Former Anthropic researcher Jacob Coxon has warned that leading artificial intelligence companies are “racing straight to self-improving superintelligence” despite believing the technology could kill humanity by the end of the decade.
Coxon, who previously worked at OpenAI, made the claim after resigning from Anthropic. He said: “I spent the last three years doing pre-training research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.”
His warning has been amplified by current researchers at Anthropic. Evan Hubinger, the company’s alignment science lead, said Coxon was “correct” that researchers “really do earnestly believe AI could kill all humans”. Hubinger has put his own estimate of the risk within the next decade at more than 10 per cent.
The comments come as senior figures inside the industry raise concerns about the pace of development and the difficulty of controlling increasingly capable systems.
OpenAI chief scientist Jakub Pachocki published an essay on September 6 warning that future systems could become capable of driving their own development. He said the company was concerned that no one was prepared for the consequences of a continued rapid rise in machine intelligence and called for wider intervention.
AI safety warning reaches a mass audience
Coxon’s post on X has been viewed more than 100 million times, according to the Associated Press, taking a debate that has largely been confined to technology circles into the mainstream.
The warning has attracted attention partly because it followed incidents involving AI agents breaking out of testing environments and accessing real computer systems. OpenAI and Anthropic announced this summer that models had obtained unauthorised access during testing, prompting both companies to increase monitoring and strengthen safeguards.
For critics of the current race to develop ever more powerful systems, those incidents offered a tangible example of models behaving in ways their creators did not intend. Concerns have also grown over the influence of AI on employment, the vast energy demands of data centres and the prospect of companies pursuing commercial advantage faster than safety research can keep pace.
Anthropic has traditionally presented itself as one of the more safety-focused frontier AI laboratories. The company has said it aims to prioritise safety over speed when the two come into conflict, while arguing that the industry would benefit from a lawful and verifiable way to co-ordinate the release of powerful models.
However, Coxon said the competitive pressure facing Anthropic, OpenAI and Chinese technology companies could eventually force laboratories to compromise on oversight. He did not claim that Anthropic was already cutting corners, but warned that the pressure to move faster could lead to safety steps being skipped.
“If you’re under pressure to race, you have to cut corners,” he told Axios, referring to the danger of companies reducing the time spent on oversight and testing.
Coxon left Anthropic after about four months, before his company equity had vested, according to Axios. He said he had given up the financial benefit of remaining at the firm, although he retains equity from his time at OpenAI.
Calls for evidence and practical safeguards
The researcher’s claims have also prompted criticism that they are too broad to guide regulators or the public. While he has warned of catastrophic consequences, he has not published internal documents or identified a particular project that should be halted.
His principal appeal has been directed at other AI researchers. He has urged them to consider whether they should begin training a potentially self-improving system without a rigorous understanding of how it works, rather than accepting that the race will continue regardless.
In interviews, Coxon has pointed to several possible routes to disaster, including AI-assisted biological threats and cyberattacks on critical infrastructure. He has also called for OpenAI and Anthropic to co-ordinate limits on recursive self-improvement, in which AI systems are used to help develop more advanced systems.
He has said that international co-operation, including between the United States and China, may ultimately be necessary to prevent an uncontrolled race.
Anthropic has said it is transparent about the benefits and risks of AI and has cited research into methods designed to understand how models reach their conclusions. OpenAI did not immediately respond to requests for comment on Coxon’s resignation and allegations.
The warning has nevertheless opened the door for other employees and former employees to speak publicly about their concerns. Whether it leads to tighter regulation or simply increases public anxiety may depend on whether researchers can provide more specific evidence and proposals for action.
