AI Procurement Risk Management Framework

Procurement of AI Systems cover sheet

Risk Management Framework for AI Procurement

Classifying and understanding risk is essential when procuring an AI system. It is important to recognize that the types and scale of risks vary from system to system. 

When AI systems are developed for high-risk domains (e.g., employment, health, education, housing, finance, public assistance, etc.), two risk indicators become highly relevant. These indicators include: 

1) the complexities within the AI system, and 

2) the impact that outcome(s) may have on human lives. 

Hence, it is imperative to determine how much risk the procuring organization is willing to accept for each system at the outset of each procurement. This act is known as establishing the risk appetite for the procurement. A well-defined risk appetite for a procurement should serve as an anchoring point throughout the procurement lifecycle to guide risk mitigation strategies and create an acceptable risk tolerance for the chosen system. 

RMF Overview

Unacceptable and High Risk Systems

The RMF for AI Procurement becomes increasingly more important when high-risk systems are at hand.





educational or vocational training, that may determine the access to education and professional course of someone’s life (e.g. scoring of exams);

employment, management of workers and access to self-employment (e.g. CV-sorting software for recruitment procedures);

financial services (e.g. denying citizens opportunity to obtain a loan);

critical infrastructures and utilities (e.g. electricity, heat, water, Internet or telecommunications access or transportation);

family planning services, including, but not limited to, adoption services or reproductive services, 

health care, including, but not limited to, mental health care, dental care or vision care; 

housing or lodging, including, but not limited to, any rental, short-term housing or lodging; 

law enforcement that may interfere with people’s fundamental rights (e.g. evaluation of the reliability of evidence);

migration, asylum and border control management (e.g. verification of authenticity of travel documents);

administration of justice and democratic processes (e.g. applying the law to a concrete set of facts);

government benefits;  

public services

Remote biometric identification systems; 

safety components of products (e.g. AI application in robot-assisted surgery).

Steps to Establishing a Risk Appetite

Every AI use case is unique.  Hence, every procurement will require a risk appetite that guides the team through the process of assessing and controlling relevant risks. The paper provides a scorecard, risk appetite matrix, convenient risk appetite statements, and suggested risk mitigation alignments to support a the responsible procurement of AI systems.

[Download Here

About the Authors

Dr. Cari Miller

Founder and Lead Researcher at The Center for Inclusive Change and Co-Founder/Board Chair of the AI Procurement Lab. Cari is a renowned AI risk expert, AI governance researcher, and consultant. She advises organizations on AI risk management, governance considerations, and responsible AI procurement. She is a certified change manager and follows an inclusive organizational design philosophy.

Gisele Waters, PhD.

Co-Founder and CEO of the AI Procurement Lab, AI governance and procurement standards builder, human-centered service designer, researcher, and culturally responsive evaluator. Gisele has built multidisciplinary guidance and tools over 25 years in education, healthcare, and information technology all threaded together by her passion for mitigating risk to vulnerable populations and communities.

Download a copy of the 

Risk Management Framework for the Procurement of AI Systems