But there is also an enormous amount of money riding on staying ahead. Both OpenAI and Anthropic are preparing for blockbuster initial public offerings. Every new capability can help justify future funding rounds, infrastructure commitments or IPOs.
As part of his proposed framework, Amodei said Anthropic would install permanent third-party reviewers inside frontier AI companies, with access to relevant tools and internal risk-assessment processes.
Reuters reported last week that a swarm of rogue OpenAI agents hijacked a German website and transformed it into a bulletin board for other AI agents, with the ChatGPT maker's officials keeping the incident under wraps as executives grappled with the fallout from the July breach of the open-source repository Hugging Face.
Many incidents in which AI agents from developers such as OpenAI have hacked or attempted to access external systems have heightened concerns over the increasing capacity of AI models and developers' ability to contain them.
Amodei said AI companies should voluntarily work together to set standards as growing numbers of U.S. lawmakers are calling for new rules to govern AI systems.
A coordinated approach, Amodei added, would enable leading U.S. AI companies to carry out necessary safety research and safeguards without putting themselves at a competitive disadvantage. Amodei said that such an approach likely would require targeted antitrust exemptions in the United States to permit collaboration in certain areas.
Amodei said any slowdown by democratically governed countries should be limited to preserving the lead U.S. AI firms hold over China, arguing that a Chinese advantage in AI would pose U.S. national security risks.
"Pacing within democracies will be limited by the lead that U.S. companies have over authoritarian regimes, chiefly the Chinese Communist Party," Amodei wrote.
Amodei called for tighter controls on advanced AI chips, model distillation and theft of model weights to prevent China from narrowing the gap.
"I believe all frontier labs should partner with government to formalize the idea of permanent embedded evaluators to better prevent and document internal alignment incidents like those that have occurred in the last few months, and to implement regulation focused on keeping capabilities in balance with safety," Amodei wrote.
QQQ•
