AI Procurement

Essential Considerations in AI Contracting

Procuring AI systems is different. It starts with defining acceptable (and unacceptable) uses of the system and traverses a path that specifies expectations toward data quality, data rights, system quality, transparency, explainability, system monitoring, ongoing risk management, and highly-specialized technical audits and inspections. 

Evaluating and controlling these factors are essential in the AI procurement and contracting processes. This is particularly important for systems designed to address needs attached to human rights, human dignity, and civil liberties. 

Our responsibility, as procuring officials of such high-risk systems, is to instantiate the values and expectations that our fellow humans deserve. This is no small task, and the playbooks are not well formed.

This playbook, based largely on the work conducted by the City of Amsterdam, provides the procurement community with a helpful set of contracting considerations when evaluating and procuring any high-risk AI system. We hope you will share this widely with achieve a world in which all high-risk AI systems are held to reasonably high standards of responsible design, development, and deployment.

Critical Definitions

Algorithmic System 

An "Algorithmic System" is software that:


An Algorithmic System does not require any form of self-learning logic (such as machine learning).

Data analysis may include the combining, cleaning, sorting, classifying, and deriving of data.

Intended Use 

Intended Use” means:


“Use” should not only define intended use(s), but also prohibited uses and foreseeable misuses, disuses, and abuses.*

(*requires routine/recurring/continuously available training for administrative users and end-users)


Decisions are:


“Decisions” are broadly interpreted – not a specific decree or decision in the legal meaning of the word.

Factual nature (i.e., municipal trash collection)

“Significant extent” means that it violates any fundamental rights, has legal consequences,  or financially impacts a citizen, user, or visitor. 

Unacceptable and High Risk Systems





educational or vocational training, that may determine the access to education and professional course of someone’s life (e.g. scoring of exams);

employment, management of workers and access to self-employment (e.g. CV-sorting software for recruitment procedures);

financial services (e.g. denying citizens opportunity to obtain a loan);

critical infrastructures and utilities (e.g. electricity, heat, water, Internet or telecommunications access or transportation);

family planning services, including, but not limited to, adoption services or reproductive services, 

health care, including, but not limited to, mental health care, dental care or vision care; 

housing or lodging, including, but not limited to, any rental, short-term housing or lodging; 

law enforcement that may interfere with people’s fundamental rights (e.g. evaluation of the reliability of evidence);

migration, asylum and border control management (e.g. verification of authenticity of travel documents);

administration of justice and democratic processes (e.g. applying the law to a concrete set of facts);

government benefits;  

public services

Remote biometric identification systems; 

safety components of products (e.g. AI application in robot-assisted surgery).

Essential Contract Clauses

The e-book contains full descriptions, details, and suggested language for each clause. 

[Download Here


Data Quality

Data Rights

System Quality


Risk Management

System Management

Audits and Inspections

3 Layers of Trust and Understanding

Explainability: Key factors that led an Algorithmic System to a particular result and can be changed to arrive at a different result. [Provided by the developer for user visibility.]

Procedural Transparency: Key factors to understand the processes, methods, and choices used in the development and deployment of the Algorithmic System. [Provided by the developer for user visibility.]

Technical Transparency: Key factors to understand the technical quality and the technical operation of the Algorithmic System. [Evaluated in confidence by a skilled AI auditor.]

See more details and context in the eBook. [Download here.]

About the Authors

Dr. Cari Miller

Founder, Principal, and Lead Researcher at The Center for Inclusive Change. She is recognized globally as one of 100 brilliant women in AI ethics and serves as the Vice Chair of the IEEE Working Group P3119, drafting an international consensus-based standard for AI procurement. She is also a doctoral candidate at Wilmington University, conducting research in AI governance and ethics.

Gisele Waters, PhD.

Founder and Lead researcher at Engineering Hearts® exploring the nature of building human-centered artificial intelligence. She is recognized globally as one of 100 brilliant women in AI ethics and serves as Chair of the IEEE Working Group P3119, drafting an international consensus-based standard for AI procurement. She is also a human-centered design researcher and healthcare service developer.  She also advises digital health start-ups on how to build human-centered data science using AI-enabled analytics and remote patient monitoring.  

Download a copy of 

AI Procurement: Essential Considerations in Contracting