Our position on open-weights models
Key Points
- 1Anthropic clarifies that it does not support a ban on open-weights models, viewing them as a potential public good rather than a threat to their business.
- 2The CEO emphasizes that national security risks should be addressed by restricting authoritarian access to high-end chips, curbing industrial-scale model distillation, and enforcing mandatory safety testing.
- 3The company advocates for empirical pre-release testing of all sufficiently capable models to mitigate cyber, biological, and alignment risks, regardless of whether they are closed or open-weights.
In this position statement, Anthropic CEO Dario Amodei addresses the controversy surrounding open-weights models and proposes a strategic framework for managing AI-related national security and safety risks. Amodei explicitly rejects the notion that Anthropic supports a blanket ban on open-weights models, arguing that they provide significant public utility, provided they do not possess dangerous capabilities.
Core Concerns
Amodei identifies two primary "nightmare scenarios" that inform his policy recommendations:- Geopolitical Superiority: The risk that authoritarian regimes (specifically the CCP) develop AI models superior to those of the US to achieve military dominance or enhance domestic repression.
- Safety and Misuse: The risk that highly capable models could be weaponized for biological or cyberattacks, compounded by potential alignment failures. He notes that open-weights models are harder to constrain once released, as they lack the centralized guardrails and monitoring capabilities inherent in closed-source, API-driven systems.
Proposed Policy Framework
Amodei argues that protectionist bans on open-weights models are ineffective, proposing instead a three-pronged regulatory approach:- Supply Chain Restrictions: Preventing the export of high-end compute hardware and chipmaking equipment to China. This strategy relies on the principle that scaling laws dictate performance; by restricting the available compute , the effective capability of a model is inherently limited by the relationship , where represents the scaling function. Restricting access to high-performance chips acts as a direct bottleneck on training state-of-the-art models.
- Mitigation of Industrial Distillation: Amodei identifies "distillation"—the process of training smaller, efficient models on the outputs of larger, frontier-level models—as a significant evasion tactic. Because distillation is compute-efficient, it allows actors to leverage existing state-sponsored frontier models to develop powerful local iterations, effectively narrowing the gap between foreign and domestic capabilities. He calls for targeted legal frameworks to disrupt these operations.
- Mandatory Pre-release Safety Testing: Amodei advocates for a global standard of rigorous, mandatory testing for all "sufficiently capable" models, regardless of whether they are open or closed-source. This testing should specifically evaluate risks related to biological synthesis, cyber offensive capabilities, and alignment. He acknowledges the "attacker-defender asymmetry," noting that the development of biological agents via AI could proceed faster than the development of medical countermeasures, necessitating empirical verification of safety before model release.