Anthropic chief executive Dario Amodei has called for a slowdown in the development of artificial intelligence, warning that companies must do more to prevent powerful systems being misused. His intervention was backed by OpenAI chief Sam Altman and Elon Musk, marking a rare show of agreement among leading figures in the industry.
Amodei said the rapid advance of AI was creating risks that could outpace safety measures, including the possibility that models could take unauthorised actions or be used to help develop biological weapons.
“If we build in the right way, I think the probability of something bad happening is very low,” he said. “If we build in the wrong way, the probability of something bad happening is very high.”
He acknowledged that the United States was competing with China to develop the technology, describing the geopolitical pressure as “the toughest aspect of our situation”. But he argued that progress could be slowed within existing limits, including by restricting the sale of advanced chips to authoritarian countries and strengthening protections against technology theft.
Amodei proposed that the US and China agree not to use AI to develop biological weapons or release models that could help bioterrorists. He also suggested that, in the longer term, countries should explore a limit on the speed of AI development, although he admitted that verifying compliance would be extremely difficult.
He said AI companies should permit independent third-party evaluators to examine their systems and work together on safety standards governing the release of new models. Government, he added, needed to be involved in decisions about how quickly products were launched.
Amodei also questioned why technology with potentially far-reaching consequences was being developed primarily by private companies. He said the public and elected representatives needed a formal role, through regulation and some form of joint oversight by democratic governments.
“I am concerned that one single government could abuse this technology just as easily as a single company could,” he said, arguing instead for a system of shared governance.
Political pressure for AI safeguards
Utah Republican governor Spencer Cox said the federal government should take a more active role in regulating AI, while stopping short of calling for a moratorium or a complete halt to development.
He said recent incidents involving AI systems taking actions outside their authorisation should concern everyone, particularly because those most familiar with the technology appeared to be increasingly alarmed.
“The possibility of an existential event happening is growing and growing much more rapidly than anyone projected,” Cox said. He called for government to work with technology companies on safeguards, pointing to Utah’s “pro-human AI approach” as a model.
Michigan Democratic Senate candidate Abdul El-Sayed went further, calling for a pause in frontier AI research until safety could be assured. He proposed turning AI companies into public trust organisations, with at least half of their oversight made up of democratically elected or appointed representatives.
El-Sayed also called for a public agency focused on AI safety and said certain uses should be prohibited. “AI shouldn’t be able to deny you health care. It shouldn’t be able to deny you a job. It shouldn’t be able to fire you. It shouldn’t be able to fire a weapon,” he said.
The debate comes as the technology becomes an increasingly important issue in the US midterm elections. Political analysts said voters were worried about AI taking jobs, increasing pressure on public infrastructure and contributing to household costs, while candidates from both parties were seeking to define their position on the issue.
Amodei said Anthropic would continue releasing advanced models, despite concerns about their potential dangers, but maintained that development needed stronger safeguards and greater public accountability.
