Dual-Use Foundation Models with Widely Available Model Weights Report
Executive Summary
As stated by President Biden, “Artificial Intelligence (AI) holds extraordinary potential for both promise and per il.”1 The development of increasingly advanced AI models, such as dual-use foundation models, has significantly heightened the potential risks and benefits of AI systems. Many developers provide limited or no public access to the inner workings of their advanced models, including their weights.2 In contrast, some developers, such as Meta, Google, Microsoft, Stability AI, Mistral, the Allen Institute for AI, and EleutherAI,3 have released models – though not always their most advanced models – with weights that are widely available (i.e., open to the public by allowing users to download these weights from the Internet or through other mechanisms).
Dual-use foundation models with widely available model weights (referred to in this Report as open foundation models) introduce a wide spectrum of benefits. They diversify and expand the array of actors, including less resourced actors, that participate in AI research and development. They decentralize AI market control from a few large AI developers. And they enable users to leverage models without sharing data with third parties, increasing confidentiality and data protection.
However, making the weights of certain foundation models widely available could also engender harms and risks to national security, equity, safety, privacy, or civil rights through affirmative misuse, failures of effective oversight, or lack of clear accountability mechanisms.
In October 2023 President Biden signed the Executive Order (EO) on “Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence.” Noting the importance of maximizing the benefits of dual-use foundation models with widely available model weights while managing and mitigating the attendant risks, Section 4.6 of the EO tasked the Secretary of Commerce, acting through the Assistant Secretary of Commerce for Communications and Information and in consultation with the Secretary of State, with soliciting feedback “from the private sector, academia, civil society, and other stakeholders through a public consultation process on the potential risks, benefits, other implications, and appropriate policy and regulatory approaches related to dual-use foundation models for which the model weights are widely available.”
Read More about the Dual-Use Foundation Models with Widely Available Model Weights Report
NTIA has prepared other materials to help stakeholders more easily navigate the Dual-Use Foundation Models with Widely Available Model Weights Report. Click below to learn more.
The EO further denoted that the Secretary of Commerce, through the Assistant Secretary of Commerce for Communications and Information and in consultation with the Secretary of State and heads of relevant agencies, would author a report to the President on the “potential benefits, risks, and implications of dual-use foundation models for which the model weights are widely available, as well as policy and regulatory recommendations pertaining to those models.” In fulfilment of this tasking, the National Telecommunications and In formation Administration (NTIA) published a public Re quest for Comment in February 2024 and received 332 comments in response.4 NTIA further conducted extensive stakeholder outreach, including two public events gathering input from a range of policy and technology experts. This Report and its findings are based in large part on this feedback.
This Report provides a non-exhaustive review of the risks and benefits of open foundation models, broken down into the broad categories of Public Safety; Societal Risks and Wellbeing; Competition, Innovation, and Research; Geopolitical Considerations; and Uncertainty in Future Risks and Benefits. It is important to under stand these risks as marginal risks—that is, risks that are unique to the deployment of dual-use foundation models with widely available model weights relative to risks from other existing technologies, including closed weight models and models that are not considered du al-use foundation models under the EO definition (such as foundation models with fewer than 10 billion parameters).
Finally, the Report considers under what circumstances the U.S. government should restrict the wide availability of model weights for dual-use foundation models. It evaluates a range of policy approaches, assessing their risks and benefits. And it concludes that, at the time of this Report, current evidence is not sufficient to definitively determine either that restrictions on such open weight models are warranted, or that restrictions will never be appropriate in the future.
Instead, this Report suggests that the government should actively monitor a portfolio of risks that could arise from dual-use foundation models with widely available model weights and take steps to ensure that the government is prepared to act if heightened risks emerge. Specifically, we recommend that the government:
- Collecting evidence through:
- Encouraging standards and – if appropriate –compelling audits, disclosures, and transparency for dual-use foundation models (including those without widely available model weights);
- Supporting and conducting research into the safety, security, and trustworthiness of foundation models and high-risk models, as well as their downstream uses;
- Supporting external research into the present and future capabilities and limitations of specific dual-use foundation models and risk mitigations; and
- Developing and maintaining a set of risk portfolios, indicators, and thresholds.
- Evaluating evidence through:
- Assessing the lag time between developers introducing capabilities in leading proprietary models, and those same capabilities being made available in open models;
- Developing benchmarks and definitions for monitoring and potential action if deemed appropriate; and
- Maintaining and bolstering federal government expert capabilities in technical, legal, social science, and policy domains to support the evaluation of evidence.
- Acting on evaluations through actions such as:
- Restrictions on access to models; or
- Other risk mitigation measures.
- Keep open the possibility of additional government action.
These recommendations support the ability of developers electing to make model weights widely available at this time, while bolstering the government’s ability to monitor whether future models pose risks that indicate that it may be appropriate to limit model weight availability or apply other appropriate risk mitigation measures. The Report provides high-level guidance and considerations for this recommendation.
This Report provides relatively little insight to future releases of dual-use foundation models with widely available model weights; however, the recommended action to monitor risks would help the government determine the capabilities of future dual-use foundation models with widely available model weights. Without changes in research and monitoring capabilities, this dynamic may persist: any evidence of risks that would justify possible policy interventions to restrict the availability of model weights might arise only after AI models with those capabilities, closed or open, have been developed or released.
In summary, this Report outlines a cautious yet optimistic path that follows longstanding U.S. government policies supporting widespread access to digital technologies and their benefits, while nonetheless preparing for the potential future development of models for which an alternate approach may be justified.
1 Exec. Order No.14,110 (2023).
2 We define limited access as AI models that do not give access to model weights, source code, or training data.
3 All mentions of specific companies or services in this report are referential and not intended to imply normativity, either positive or negative, regarding the company or service
4 Dual Use Foundation Artificial Intelligence Models with Widely Available Model Weights, 89 Fed. Reg. 14059 Pub (Feb. 26, 2024).