Recommendations

Active monitoring by the federal government of the continued risks arising from dual-use foundation models with widely available model weights should involve a risk-specific risk management approach that includes three steps:

1. Collect evidence about the capabilities, risks, and benefits of the present and future ecosystem of dual-use foundation models with widely available model weights, and monitoring specific model-based and downstream indicators of risk for potential risk cases, as well as the difference in capabilities between open foundation models and proprietary models;
2. Evaluate that evidence by comparing indicators against specified thresholds, to determine when risks are significant enough to change the federal government’s approach to open-weight foundation model governance; and when appropriate,
3. Act on those evaluations by adopting policy and regulatory measures targeted appropriately across the AI value chain.

The United States government does not currently have the capacity to monitor and effectively respond to many of the risks arising from foundation models. A significant component of our recommendation is to increase the government’s capacity for evidence gathering, agile decision-making, and effective action.

Step 1: Collect Evidence

We recommend that the federal government take steps to ensure that policymakers have access to a high-quality evidence base upon which to assess policy approaches to dual-use foundation models with widely available model weights going forward.²⁶¹ To develop and promote that evidence base, the federal government should:

1. Encourage, Standardize, and, if Appropriate, Compel Auditing and Transparency for Foundation Models

   It is difficult to understand the risks of dual-use foundation models without transparency into AI model development and deployment, including downstream uses. To the extent reasonable, the federal government should standardize testing and auditing methods, which may vary based on the capabilities, limitations, and contexts of use of particular models and systems. The capabilities and limitations of closed-weight foundation models are currently good indicators of the potential future capabilities and limitations of open-weight models. The federal government should encourage, and where appropriate and where authority exists, require either independent or government audits and assessments of certain closed-weight foundation models²⁶² – especially closed-weight models whose capabilities exceed those of advanced dual-use foundation models with widely available model weights and can therefore serve as a leading indicator of the future capabilities of those models. For instance, the U.S. AI Safety Institute, housed in the National Institute for Standards and Technology, plans to perform pre-and post-deployment safety tests of leading models. This work should help the federal government understand and predict the risks, benefits, capabilities, and limitations of dual-use foundation models with widely available model weights. The federal government should also aim to enable independent researcher access, in addition to U.S. AI Safety Institute access, to certain closed-weight foundation models, including downstream effects of AI on the information individuals receive and how it affects their behavior. This will help assess the risks and benefits that could arise from future models.

   As model capabilities and limitations change, so will the appropriate testing and auditing procedures. The Unit ed States should stay actively engaged in updating those methods and procedures. The federal government should establish criteria to define the set of dual-use foundation models that should undergo pre-release testing before weights are made widely available, with the results of such testing made publicly available to the extent possible. This evaluation should be done with the complete spectrum of model uses in mind, from deployment by model developers to distribution on platform/hosting intermediaries to specific business uses.

2. Support and Conduct Safety, Security, and Trustworthiness Research into Foundation Models and High Risk Models, including Downstream Uses

   1. PERFORM INTERNAL GOVERNMENT RESEARCH

      The U.S. government should engage in its own active re search and analysis. The government should also con tinue to build capacity for a broad array of expertise and functions to conduct this research. Work being done by a variety of agencies in their respective areas of subject matter expertise could provide better insight into potential gaps that may need to be filled to promote an open ecosystem while addressing risks. For example, the U.S. Copyright Office is undergoing a comprehensive initiative to examine copyright issues raised by AI.²⁶³ The Department of Energy’s Frontiers in AI for Science, Security, and Technology (FASST)²⁶⁴ initiative plans to leverage the departments’ supercomputers to provide insights into dual-use foundation models and better assess potential risks. The outcome of initiatives such as these could create a better sense of the state of play in different fields (e.g., for the U.S. copyright system, a more comprehensive under standing of the interplay between the “fair use” doctrine and the use of copyrighted works without permission from the rights holder to train AI models). Consequently, any research and data gathering should, where appropriate, involve collaboration between relevant government agencies.

      Research into foundation models should not just include technical aspects of the models. It should also cover areas of research such as the effects of AI on human actions, privacy, legal ramifications, and downstream effects o f dual-use foundation models. This research should also address, for instance, the potential ability of these models t o increase CBRN risks, in particular, bio risks, as well as cybersecurity concerns, and risks of human deception.

   2. SUPPORT EXTERNAL RESEARCH

      The federal government should support external research on the risks and benefits related to dual-use foundation models. Research into available technical and non-technical mitigations for risks arising from dual-use foundation models with widely available model weights is also important to prioritize. This could include research into model explainability/interpretability and approach other approaches identified by research communities. Support could take the form of direct research grants, including through the National AI Research Institutes, or it could be provided by prioritizing such research through compute resource support programs like the proposed NAIRR.

3. Develop and Maintain Risk Portfolios, Indicators, and Thresholds

   The U.S. government should identify specific risks, and then, for each identified risk, maintain one or more risk indicators. These can be technical indicators, such as multi-modal capabilities or the ability of AI agents to manipulate the external environment, or measurements of confabulation or racial bias. They could also be societal indicators, such as the breadth of adoption of a particular AI system or the availability of certain physical materials which could be used in conjunction with AI to create a threat.

   One important class of metrics for open-weight foundation models is leading indicators. These are indicators of the risks, benefits, and capabilities that open-weight foundation models will – but do not currently – possess. It is important that the government maintain robust leading indicators of model capabilities, because harms from open models are difficult to undo once the weights are released. While the existing capabilities of closed-weight models are one leading indicator of the future capabilities, risks, and benefits of open-weight foundation models, they are not the only ones. Tracking the relative rate of advances between open- and closed-weight models, for example by comparing their performance on complex tasks over time, would help identify when a given open-weight model is poised to catch up to or surpass the capabilities of an existing closed-weight model. By creating these metrics, the government can better prepare for future risks and take advantage of future benefits as these technologies continue to rapidly evolve.

   To actively monitor the open-weight foundation model ecosystem, the federal government should maintain a portfolio of risk cases, including unlikely risks and sociotechnical risks, that might arise from future open foundation models. Each such risk should be accompanied by (i) one or more leading indicators of risk, which can be social and/or technological, (ii) thresholds for each indicator, and (iii) a set of potential policy responses that could mitigate the risk. Benefit indicators should also be taken into account when risk-benefit calculations are important. When the indicator(s) meet the threshold(s), the government should consider intervening with one or more policy responses. An example of this scenario is given in the Appendix.

   The choice of thresholds and potential policy responses should weigh current and predicted future technical capabilities, relevant legal considerations, and downstream impacts.

   In establishing thresholds and conducting assessments, the government should recognize that the evidence base for restrictions on dual-use foundation models with widely available model weights is evolving. The benefits that such models produce should be fully considered in establishing those thresholds, as well as the legal and international enforcement challenges in implementing restrictions.²⁶⁵ Additionally, consideration should be given to whether each risk is better addressed with interventions in downstream pathways through which those risks materialize rather than in the availability of model weights.²⁶⁶

Step 2: Evaluate Evidence

Using a broad evidence base and specific risk indicators, the federal government should assess whether the marginal risks from open-weight models in a particular sector or use case warrants government action. Specifically, the federal government should:

1. Assess the Difference in Capabilities, Limitations, and Information Content between Closed and Open Models

   The government should assess and monitor the length of the “policymaking runway”: the length of time between when leading closed models achieve new capabilities and when open-weight models achieve those same capabilities, along with a wider set of indicators including persistent limitations, and information about training data and information content associated with open weight models.

   Once a capability appears in an open-weight model, it may be impossible to wholly remove that capability from the open-weight foundation model ecosystem. Therefore, restrictions on open-weight models can be most effective only before a particular capability is released in an open weight model. Likewise, a rich understanding of limitations can help downstream integrators make informed choices when selecting open models.

   Many factors may affect the policymaking runway, and its length will affect the speed with which policymakers will need to respond to changes in capabilities, limitations, and information content of open models available in the open model ecosystem.

2. Develop benchmarks and definitions for monitoring and action.

   There are a range of factors that should be considered when developing monitoring benchmarks and definitions, not only those listed in the EO definition of dual-use foundation models. Numerical measures such as the number of floating-point operations used in training provide rough estimates of model capabilities, and can be used as a first step to distinguish models that deserve further scrutiny. But to properly calibrate monitoring and policy interventions to the appropriate models, the US government should developing benchmarks and definitions for model capabilities that incorporate other factors as well. One rea son for this is that, while numerical measures like the number of parameters/weights or floating point operations per second (FLOPS) are often related to a model’s technical capabilities,²⁶⁷ advances in algorithms, architectures, processors, and the complexities posed by multi-modal models may gradually cause any numerical metric to become outdated. For instance, the Executive Order refers to “tens of billions of parameters” in the definition of dual-use foundation model. However, Meta’s Llama 3 8B, which did not exist at the time the Executive Order was written and does not have enough parameters to meet this definition, outperforms LLama 2 70B,²⁶⁸ which does meet the definition, on a number of benchmarks.²⁶⁹ With computing capabilities increasing exponentially over time,²⁷⁰ it is quite possible that personal computers will someday be able to train highly capable and generalizable models comparable to today’s most advanced foundation models.²⁷¹

   Furthermore, the risks and benefits of AI arise in complicated social and technical ways, which depend on the type of information processed by the model and the potential set of use cases. Evo, a state-of-the-art AI biological design tool that can work with proteins, DNA, and RNA,²⁷² seems to fit most of the requirements for a dual-use foundation model.²⁷³ However, some biological design tools currently only involve approximately hundreds of millions of parameters – far less than in the dual-use foundation model definition. Many text-to-image and text-to-video models do not require more than 10 billion parameters.²⁷⁴ A giant model is not required to make a deepfake video – it can be done on a personal computer.²⁷⁵

   In addition to the number of parameters, there are many other features that make AI models potentially powerful and which may be useful in establishing benchmarks and definitions for monitoring and action. Policymakers and researchers should take into consideration the following non-exhaustive list of factors. The relative importance of each factor will vary depending on the situation:

   1. Number of parameters
   2. Computing resources required to train a model
   3. Training data – dataset size and quality, nature and confidentiality of the data, difficulty of reproducing the data.
   4. Model architecture and training methods
   5. Versatility – the types of tasks a model can perform
   6. Potential risks – explicitly identified use cases that create specific harms
   7. Access and adoption – the number of people, organizations, and systems that use or are affected by the model
   8. Emergence – the ability of a model to perform tasks that it was not explicitly trained for
   9. Evaluated capabilities – performance on particular tasks, including non-technical tasks such as AI-human interactions²⁷⁶
   10. Information modalities – the types of information the model can process, such as image, text, genetic data²⁷⁷, biometric data²⁷⁸, real-world sensing²⁷⁹ or combinations of multiple types.

3. Maintain and Bolster Cross-disciplinary Federal Government Capacity to Evaluate Evidence

   Effective monitoring, assessment, and decision-making will require cross-disciplinary expertise and resources. The U.S. government should encourage and hire this type of talent. Technical specialists and access to AI models will be necessary to stay current on model capabilities. But social scientists will also be necessary to understand the economic and social effects of dual-use foundation models with widely available model weights. Legal experts, including privacy, First Amendment, copyright, foreign pol icy, as well as human and civil rights scholars, should be consulted on the legal and constitutional implications of intervening or failing to intervene. Domestic and international policy analysts will help navigate the complexities of government decision-making. The government has made significant strides in increasing the Federal AI workforce through the AI Talent Surge launched by EO 14110. The United States should continue that trend by hiring top talent across the fields that foster AI-related skills.²⁸⁰ Particular care should be taken to maintain effective cross-agency collaboration because the impacts of dual-use foundation models do not fit neatly in any one category.

Step 3: Acting on Evaluations

Given the varied nature of risks that foundation models can and will pose, the government should maintain the ability to undertake interventions, which should be considered once the risk thresholds described above are crossed such that the marginal risks substantially outweigh the margin al benefit. These interventions include restrictions on access to models (including model weights) and other risk mitigation measures as appropriate to the specific con text when restrictions on widely available model weights are not justified or legally permissible.

1. Model Access Restrictions

   One broad category of such interventions involves restricting access to, or requiring pre-release model licensing for, certain classes of dual-use foundation models or systems. At one end of this category is complete restriction of a model from being publicly distributed, including model weights and API access. A less extreme step would involve restricting the open sharing of model weights and allowing public access only to hosted models. These restrictions would impose substantial burdens on the open-weight model ecosystem and should require significant evidence of risk. There are many different ways to implement a structured access program that restricts access to model weights,²⁸¹ where government could set guidelines “for what capabilities should be made available, in what form, and to whom.”²⁸² The government could also mandate that intermediary AI platforms ensure that restricted weights are not available on their platforms or are only available in select instances. These restrictions could potentially be effectuated through existing statutory authorities (such as the Export Administration Act) or through Congressional action, though this Report does not consider questions of legal authority in detail.

   Any consideration of the appropriate scope or nature of these restrictions would require legal and constitutional analysis.²⁸³ Intellectual property considerations, which are not the principal focus of this Report, would also inform the question of whether, and how far, to restrict.

   Importantly, the effects of AI and potential causes of AI risk are not bound to any single country, and the effectiveness of restrictions on the distribution of model weights depends in significant part on international alignment on the appropriate scope and nature of those restrictions. The federal government should prioritize international collaboration and engagement on its policy concerning the governance of dual-use foundation-models with widely available model weights.

   The United States should also retain the ability to promote certain types of openness in situations that have the potential to pose risk, but for which there is not enough information. This could include structured access for researchers,²⁸⁴ further information gathering on the part of the U.S. government, or funding for specific risk research.

2. Other Risk Mitigation Measures

   Because the risks and benefits of dual-use foundation models are not solely derived from the model itself, appropriate policy measures may not concern the model weights specifically, depending on the nature of the risks and benefits.²⁸⁵ The government should maintain the ability to respond with a wide range of risk mitigations in accordance with its legal authority. The foundation model ecosystem has many components, and in many cases the most effective risk reduction will happen downstream of the model weights. It is important to note that several enforcement agencies have indicated that their authorities apply to the latest developments in AI technology, for example to address discrimination and bias.²⁸⁶

   Whether and how regulations apply throughout the AI stack is beyond the scope of this Report, but it is an area worth exploring. These mitigations will likely depend on the specific risk. For example, in cases where dual-use foundation models with widely available model weights enable creation of dangerous physical objects, restrictions on physical materials may be warranted.

   Firm data privacy protections should be developed and adapted as foundation models continue to interact with, and draw data from, progressively larger data sets, processed at higher velocities, that touch on more parts of Americans’ lives. Other mitigation measures might include better content moderation on online platforms to limit illegal or abusive generated content, improved spear-phishing filters for emails, user interface designs to highlight possible misinformation and limited accessibility to CBRN datasets. Effective mitigations could also include making potentially impacted systems more robust and resilient to harmful effects of AI. This could include minimizing the reach of disinformation campaigns, and providing support resources for human victims of AI-generated harms. Ultimately, a combination of education, experience, research, and proactive efforts by model creators will likely be necessary to help mitigate a broad array of risks.

Additional Government Action

While actively monitoring risks, the government should also support openness in ways that enhance its benefits. This should include incentivizing social, technical, economic, and policy research on how to ensure open foundation models promote human well-being. Government agencies may be able to use their authorities or subject matter expertise to promote an open ecosystem while addressing risks. Fiscal policy could also be used to support open foundation models, for instance through subsidies for open models. One promising subsidy-based approach is the NAIRR, which has embedded open source and open science principles into its workplan.

The U.S. government should also continue leading international diplomacy and norm-setting efforts around open foundation models. This should include engagement with a broad spectrum of international partners and fora to ensure the benefits of open artificial intelligence are shared, while limiting the ability of bad actors to cause harm. The U.S. government should also work with its allies to ensure that the uses of open-weight foundation models support the principles of democratic representation and freedom, rather than autocracy and oppression.

Next: Conclusion