A leading Anthropic researcher has warned that artificial intelligence could potentially become an existential threat to humanity, estimating that there is more than a 10% chance AI could kill all humans within the next decade.
The comments followed the resignation of Jacob Coxon, a 27-year-old pretraining researcher who previously worked at OpenAI and Anthropic. In a post on social media, Coxon said AI developers genuinely believe advanced systems could eventually destroy humanity and argued that this is not merely a publicity strategy.
Coxon said he no longer believes that any single AI lab can safely develop self-improving AI, regardless of how carefully it approaches the technology. He accused major AI companies of “racing straight to self-improving superintelligence and gambling with our lives.”
Evan Hubinger, Alignment Science Lead at Anthropic, responded that Coxon’s characterization was accurate. Hubinger said he personally puts the probability of AI killing all humans within the next decade at more than 10%.
He also said Anthropic is trying to develop AI safely but acknowledged that the company does not yet have a plan to solve alignment for superintelligence or clear evidence that it is on track to do so.
According to Hubinger, the biggest concern is not necessarily current systems such as Claude. He said the greater risk could come from recursive self-improvement, in which an AI becomes increasingly intelligent by improving or rewriting aspects of itself.
The comments have prompted debate about differences in AI safety thinking across major technology companies. UCLA adjunct professor Arun Rao said the episode reflects a wider ideological split within the AI industry, with more effective-altruism-influenced “doomer” views, in his assessment, present at Anthropic than at several other major labs.
The controversy has also attracted attention from Washington. US Congressman Ted Lieu cited the comments while calling for the passage of the bipartisan AI Kill Switch Bill, arguing that warnings from researchers inside frontier AI companies strengthen the case for tougher safeguards.
The exchange reflects an increasingly urgent debate about the risks of self-improving AI and whether existing safety measures can keep pace with the rapid development of increasingly powerful systems.
