Anthropic CEO Dario Amodei is calling for an immediate slowdown in the pace of AI development, warning of potentially devastating consequences in a matter of months otherwise.
- OpenAI CEO Sam Altman quickly agreed that the industry needs to slow the pace of frontier-model advances and take more steps on safety.
Why it matters: Amodei pulled no punches in a new essay, cautioning that swarms of rogue AI agents could take over the internet in as little as six months from now.
The big picture: This is the week that the AI safety debate broke into the public consciousness, driven by an Anthropic employee's very public resignation and warning of possible doom.
- Amodei's essay isn't necessarily a direct response, but it will have much the same effect amid a sudden surge of public and congressional outrage.
What they're saying: "We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain," Amodei wrote.
- "The measures I propose to advance the frontier at a safe pace will not be easy. But I believe we owe it to humanity to try."
Threat level: He also specifically referenced the recent OpenAI-Hugging Face rogue agent incident, which he said could have been far worse.
- "Given the accelerating rate of AI capability development, it's my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails," he warned.
Zoom out: Amodei said Anthropic would unilaterally take one key safety measure — giving external evaluators employee-level access to ensure safety procedures and report incidents.
- He called on the rest of the industry to do the same.
- He also called for coordination among democratic countries to establish safety standards, and an effort for those countries to coordinate with authoritarian governments as well.
Zoom in: Anthropic executives followed Amodei's essay with a call for legislative and regulatory responses.
- "The government has a critical role to play here, including blocking the sale of the most advanced chips to adversarial nations like China, enacting a national law requiring testing of frontier models, with the power to block the most advanced models that prove to be unsafe," public policy chief Sarah Heck wrote on X.
The intrigue: SpaceXAI's Elon Musk quickly endorsed Amodei's call to action.
- "Dario is right," he wrote on X. (Anthropic is a major customer of Musk's for data center capacity.)
- OpenAI's Altman later posted that "I agree with Dario that we need to pace the frontier," and said his company would also give access to external evaluators.
Reality check: So much money is at stake in the AI industry that it can be hard to imagine companies restraining themselves, especially if they're not sure competitors will do the same.
- Just last month, OpenAI said it would slow development of its newest model, Astra, due to cybersecurity concerns.
- Anthropic itself has gone back and forth on the value of unilaterally slowing development.
In fact, Amodei's critics suggested the essay was as much about market position as safety.
- "Dario makes the case to stop open source and concentrate enormous technological and economic power with Anthropic," venture capitalist Chamath Palihapitiya wrote on X.
The bottom line: Amodei's essay is likely to have a significant impact on the safety and regulatory debate.
- But whether anyone acts, or if the industry simply continues hurtling toward an incredibly uncertain future, remains to be seen.
Editor's note: This story has been updated with additional context throughout.