As part of developing an AI Risk Management Framework, the US National Institute for Science and Technology (NIST) has published a draft report identifying three main categories of risk of which those designing, developing and using AI should be aware.

Those categories of risk are:

1. Technical design attributes

This refers to the reliability, accuracy and robustness of the systems being used. Most relevant to AI developers and designers, evaluation criteria should be used to assess accuracy and sources of error. Particular consideration should be exercised when applying AI to new data, and deemed standards of safety and determinations of security must be addressed.

2. How AI systems are perceived

In general, assessments of an AI system or a decision deriving from AI ought to be able to be scrutinised by a human.  This is an important aspect of gaining public trust. AI systems and decisions deriving from AI therefore require:

• Transparency: whether the AI's output is sufficient to evaluate compliance.
• Explainability: whether the systems can be easily understood, the outcome of an AI system can be properly explained and communicated, and how issues such as bias are being addressed.
• Interpretability: whether the systems, including their use of data, and any decisions being made can be used to make a meaningful decision.

Interpretability is associated with simple representations whereas transparency may create information overload.  Information must be explained in such a way that there can be a coherent understanding of the AI system's use and any decisions derived in context.  There is a (potentially difficult) balance to be struck and there will be competing factors (such as protecting proprietary rights).  See our previous articles on How to ensure transparent and accountable algorithmic decision making and The need for AI transparency in the public sector.

3. Guiding policies and principles

This addresses societal perceptions of the trustworthiness of AI. Key risks AI developers and purchasers should consider are privacy (particularly critical when processing personal data), fairness, justice, equality, bias, accountability and good governance.  These can be difficult to measure, at least consistently across different locations where the AI is deployed, as they are context dependent . Further issues may arise where ensuring fairness requires violating privacy and vice versa.

Developing NIST's AI framework

The categories of risk listed above are not intended to be exhaustive.   But by identifying the broad categories of risk NIST hopes to help develop an "overarching approach" which forms part of NIST's development of an AI Risk Management Framework.  The Framework is "is intended for voluntary use and to improve the ability to incorporate trustworthiness considerations into the design, development, use, and evaluation of AI products, services, and systems".  

NIST development of an AI Risk Management Framework is, in their words, "an ideal opportunity to advance those discussions and forge agreements across organizations and internationally to the benefit AI design, development, use, and evaluation."  This reflects the international aspect of discussions about AI.  For example, the EU proposals for AI legislation is drafted with the intention of setting global standards for regulating AI.  So whilst any risk assessment (whether required by law or not) must be based on current and expected circumstances, an eye also should be had on international developments on how to best identify and manage the risks arising from AI. 

This article was written by Eve Jenkins and Tom Whittaker