The integration of artificial intelligence (AI) into public sector procurement presents significant opportunities and challenges. As governments (and the pensions industry) increasingly turn to AI to enhance efficiency and decision-making, it is crucial that public bodies understand the considerations relevant when procuring AI and these will be relevant to public sector bodies considering procurement of AI in relation to their pension provision. 

The recently published paper by the Ada Lovelace Institute (the “AL Paper”) considers this issue.

General Considerations for Public Sector Bodies

Legal Framework and Compliance

Public sector procurement of AI must adhere to existing legal frameworks, including data protection laws, procurement regulations, sector-specific legislation and obligations of equality and fairness. Conducting an impact assessment of obtaining and using AI technology is one way to help achieve this.

Ethical Considerations

Ethical considerations are central in the procurement and use of AI, especially for public sector entities. The AL Paper advises embedding ethical principles into the procurement process, addressing bias, discrimination, and transparency. Public bodies should obtain disclosures from AI suppliers to ensure their technology does not act discriminatorily and assess this information against the Alan Turing Institute’s ‘FAST Track Principles’ of Fairness, Accountability, Sustainability, and Transparency.

Risk Management

Effective risk management is essential in AI procurement. While AI offers significant benefits, it also poses risks, such as further exposing the purchaser to cyber security threats. The AL Paper highlights the importance of robust risk assessment frameworks to address these challenges. Public bodies should take steps to understand how AI uses data and also obtain the AI supplier’s cyber policies and procedures. The National Cyber Security Centre provides guidance on assessing the cyber-risk posed by AI technology. For more information on this in the context of the pensions industry, see our recent blog on the relationship between cyber security and AI.

Considerations for Public Sector Pension Schemes

Public sector pension schemes, such as the Local Government Pension Scheme (LGPS), and public sector bodies involved with them, should be aware of the general considerations mentioned above when procuring AI. Additionally, they should note pensions-specific factors such as the following under the General Code:

The General Code

The General Code requires schemes to identify risks and manage them in a proportionate manner. This includes:

• Assessing risks “affecting operational resilience, including where those risks belong to service providers” (e.g. including an AI supplier).
• Consider “running a tender process when appointing… service providers, and commit[ting] enough time and resources [to this process]”, in addition to “carry[ing] out due diligence as part of the appointment process”. 
• Once the AI supplier is providing services, the Trustees should consider obtaining “assurance that [the AI supplier is] meeting their own standards for internal controls” (e.g. that any updates to the AI technology continue to meet the non-bias/discrimination requirements). 

Note that the governing body retains “ultimate accountability”, even where responsibilities have been delegated to a service provider. 

Conclusion

The procurement of AI in the public sector offers transformative potential but requires careful consideration of legal, ethical, and risk management factors. 

We work closely with AI specialists from our firm and in the industry to understand how the technology is likely to impact our clients. Please do reach out to Chris Brown, Callum Duckmanton, or your usual Burges Salmon contact if you would like to discuss the procurement of AI services.  

For the latest updates on AI law, regulation, and governance, see our AI blog.

This article was written by Callum Duckmanton (Solicitor) and Chris Brown (Partner), both in our Pensions & Lifetime Savings Team.