Executive Summary
Artificial intelligence (AI) systems are rapidly becoming part of the fabric of everyday American life. From customer service to image generation to manufacturing, AI systems are everywhere.
Alongside their transformative potential for good, AI systems also pose risks of harm. These risks include inaccurate or false outputs; unlawful discriminatory algorithmic decision making; destruction of jobs and the dignity of work; and compromised privacy, safety, and security. Given their influence and ubiquity, these systems must be subject to security and operational mechanisms that mitigate risk and warrant stakeholder trust that they will not cause harm.
Commenters emphasized how AI accountability policies and mechanisms can play a key part in getting the best out of this technology. Participants in the AI ecosystem – including policymakers, industry, civil society, workers, researchers, and impacted community members – should be empowered to expose problems and potential risks, and to hold responsible entities to account.
AI system developers and deployers should have mechanisms in place to prioritize the safety and well-being of people and the environment and show that their AI systems work as intended and benignly. Implementation of accountability policies can contribute to the development of a robust, innovative, and informed AI marketplace, where purchasers of AI systems know what they are buying, users know what they are consuming, and subjects of AI systems – workers, communities, and the public – know how systems are being implemented. Transparency in the marketplace allows companies to compete on measures of safety and trustworthiness, and helps to ensure that AI is not deployed in harmful ways. Such competition, facilitated by information, encourages not just compliance with a minimum baseline but also continual improvement over time.
Read More about the AI Accountability Policy Report
NTIA has prepared other materials to help stakeholders more easily navigate the Artificial Intelligence Accountability Policy Report. Click below to learn more.
To promote innovation and adoption of trustworthy AI, we need to incentivize and support pre- and post-release evaluation of AI systems, and require more information about them as appropriate. Robust evaluation of AI capabilities, risks, and fitness for purpose is still an emerging field. To achieve real accountability and harness all of AI’s benefits, the United States – and the world – needs new and more widely available accountability tools and information, an ecosystem of independent AI system evaluation, and consequences for those who fail to deliver on commitments or manage risks properly.
Access to information by appropriate means and parties is important throughout the AI lifecycle, from early development of a model to deployment and successive uses, as recognized in federal government efforts already underway pursuant to President Biden’s Executive Order Number 14110 on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence of October 30, 2023 (“AI EO”). This information flow should include documentation about AI system models, architecture, data, performance, limitations, appropriate use, and testing. AI system information should be disclosed in a form fit for the relevant audience, including in plain language. There should be appropriate third-party access to AI system components and processes to promote sufficient actionable understanding of machine learning models.
Independent evaluation, by appropriate means and parties is important throughout the AI lifecycle, from early development of a model to deployment and successive uses, as recognized in federal government efforts already underway pursuant to President Biden’s Executive Order Number 14110 on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence of October 30, 2023 (“AI EO”). This information flow should include documentation about AI system models, architecture, data, performance, limitations, appropriate use, and testing. AI system information should be disclosed in a form fit for the relevant audience, including in plain language. There should be appropriate third-party access to AI system components and processes to promote sufficient actionable understanding of machine learning models.
Consequences for responsible parties, building on information sharing and independent evaluations, will require the application and/or development of levers – such as regulation, market pressures, and/or legal liability – to hold AI entities accountable for imposing unacceptable risks or making unfounded claims.
The AI Accountability Policy Report conceives of accountability as a chain of inputs linked to consequences. It focuses on how information flow (documentation, disclosures, and access) supports independent evaluations (including red-teaming and audits), which in turn feed into consequences (including liability and regulation) to create accountability. It concludes with recommendations for federal government action, some of which elaborate on themes in the AI EO, to encourage and possibly require accountability inputs.
In April 2023, the National Telecommunications and Information Administration (NTIA) released a Request for Comment (“RFC”) on a range of questions surrounding AI accountability policy. The RFC elicited more than 1,400 distinct comments from a broad range of stakeholders. In addition, we have met with many interested parties and participated in and reviewed publicly available discussions focused on the issues raised by the RFC.
Based on this input, we have derived eight major policy recommendations, grouped into three categories: Guidance, Support, and Regulations. Some of these recommendations incorporate and build on the work of the National Institute of Standards and Technology (NIST) on AI risk management. We also propose building federal government regulatory and oversight capacity to conduct critical evaluations of AI systems and to help grow the AI accountability ecosystem.
While some recommendations are closely linked to others, policymakers should not hesitate to consider them independently. Each would contribute to the AI accountability ecosystem and mitigate the risks posed by accelerating AI system deployment. We believe that providing targeted guidance, support, and regulations will foster an ecosystem in which AI developers and deployers can be properly held accountable, incentivizing the appropriate management of risk and the creation of more trustworthy AI systems.
1. Audits and auditors: Federal government agencies should work with stakeholders as appropriate to create guidelines for AI audits and auditors, using existing and/or new authorities. This includes NIST’s tasks under the AI EO concerning AI testing and evaluation and other efforts in the federal government to refine guidance on such matters as the design of audits, the subject matter to be audited, evaluation standards for audits, and certification standards for auditors.
2. Disclosure and access: Federal government agencies should work with stakeholders to improve standard information disclosures, using existing and/or new authorities. Greater transparency about, for example, AI system models, architecture, training data, input and output data, performance, limitations, appropriate use, and testing should be provided to relevant audiences, including in some cases to the public via model or system cards, datasheets, and/or AI “nutrition labels.” Standardization of accessible formats and the use of plain language can enhance the comparability and legibility of disclosures. Legislation is not necessary for this activity to advance, but it could accelerate it.
3. Liability rules and standards: Federal government agencies should work with stakeholders to make recommendations about applying existing liability rules and standards to AI systems and, as needed, supplementing them. This would help in determining who is responsible and held accountable for AI system harms throughout the value chain.
4. People and tools: Federal government agencies should support and invest in technical infrastructure, AI system access tools, personnel, and international standards work to invigorate the accountability ecosystem. This means building the resources necessary, through existing and new capacity, to meet the national need for independent evaluations of AI systems, including:
- Datasets to test for equity, efficacy, and other attributes and objectives;
- Computing and cloud infrastructure required to conduct rigorous evaluations;
- Legislative establishment and funding of a National AI Research Resource;
- Appropriate access to AI systems and their components for researchers, evaluators, and regulators, subject to intellectual property, data privacy, and security- and safety-informed protections;
- Independent evaluation and red-teaming support, such as through prizes, bounties, and research support;
- Workforce development;
- Federal personnel with the appropriate sociotechnical expertise to design, conduct, and review evaluations; and
- International standards development (including broad stakeholder participation).
5. Research: Federal government agencies should conduct and support more research and development related to AI testing and evaluation, tools facilitating access to AI systems for research and evaluation, and provenance technologies, through existing and new capacity. This investment would move towards creating reliable and widely applicable tools to assess when AI systems are being used, on what materials they were trained, and the capabilities and limitations they exhibit. The establishment of the U.S. AI Safety Institute at NIST in February 2024 is an important step in this direction.
6. Audits and other independent evaluations: Federal agencies should use existing and/or new authorities to require as needed independent evaluations and regulatory inspections of high-risk AI model classes and systems. AI systems deemed to present a high risk of harming rights or safety – according to holistic assessments tailored to deployment and use contexts – should in some circumstances be subject to mandatory independent evaluation and/or certification. For some models and systems, that process should take place both before release or deployment, as is already the case in some sectors, and on an ongoing basis. To perform these assessments, agencies may need to require other accountability inputs, including documentation and disclosure relating to systems and models. Some government agencies already have authorities to establish risk categories and require independent evaluations and/or other accountability measures, while others may need new authorities.
7. Cross-sectoral governmental capacity: The federal government should strengthen its capacity to address cross-sectoral risks and practices related to AI. Whether located in existing agencies or new bodies, there should be horizontal capacity in government to develop common baseline requirements and best practices, and otherwise support the work of agencies. These cross-sectoral tasks could include:
- Maintaining registries of high-risk AI deployments, AI adverse incidents, and AI system audits;
- With respect to audit standards and/or auditor certifications, advocating for the needs of federal agencies and coordinating with audit processes undertaken or required by federal agencies themselves; and
- Providing evaluation, certification, documentation, coordination, and disclosure oversight, as needed.
8. Contracting: The federal government should require that government suppliers, contractors, and grantees adopt sound AI governance and assurance practices for AI used in connection with the contract or grant, including using AI standards and risk management practices recognized by federal agencies, as applicable. This would ensure that entities contracting with the federal government or receiving federal grants are enacting sound internal AI system assurances. Such practices in this market segment could accelerate adoption more broadly and improve the AI accountability ecosystem throughout the economy.