Jacob Coxon, a researcher who spent three years working on pretraining at both OpenAI and Anthropic, has resigned from Anthropic with a stark warning: frontier AI laboratories are racing toward systems they may not be able to control. In his resignation announcement, Coxon said both companies are "racing straight to self-improving superintelligence and gambling with our lives."
Coxon's departure gained additional weight after Evan Hubinger, Anthropic's Alignment Science Lead, publicly stated that he personally believes there is a greater than 10% chance that AI could kill all humans within the next decade. Hubinger was quick to clarify that this is his own estimate, not an official Anthropic forecast, and that he considers the risk from present-day models to be relatively low. His concern centers on future superintelligence emerging through recursive self-improvement, where AI systems increasingly help design and improve their successors.
The race itself is the problem
Coxon is not simply arguing that Anthropic ignores safety. In his resignation statement, he acknowledged that people at Anthropic understand the potential stakes but remain locked in a competitive dynamic that rewards moving faster because another laboratory might otherwise reach superintelligence first. "They are racing straight to self-improving superintelligence and gambling with our lives," Coxon wrote.
The researcher has decided to leave the AI industry entirely because he no longer wants to participate in building systems that could become uncontrollable. He told the Wall Street Journal that under aggressive scenarios, things could be "out of control" by the end of next year. That timeline remains Coxon's personal assessment, but his argument raises a harder governance question: a company can take safety seriously and still contribute to a dangerous race if competitive pressure makes slowing down feel impossible.
Hubinger's 10% estimate exposes the contradiction
Hubinger's response turned the resignation into a wider debate about frontier AI. In a post on X, he said, "Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to."
Anthropic has publicly said that recursive self-improvement is not inevitable, but could arrive sooner than institutions are prepared for and could increase the risk of humans losing control. The contradiction is difficult to ignore: researchers are trying to solve alignment while the systems they study continue becoming more capable.
Competition may be harder to solve than alignment
Coxon's resignation ultimately points beyond any single company. If leading laboratories believe that slowing down could hand an advantage to a rival, serious internal concern may not translate into slower development. AI researcher Gary Marcus, in comments reported by Techmeme, said he did not agree with every part of Coxon's argument but found it "compelling and informed from the inside" and said it deserved to be heard.
That is a useful counterweight because claims about extinction probabilities and superintelligence timelines remain deeply uncertain. Anthropic can invest in alignment research, publish risk reports, and build safeguards, while still operating in a market where OpenAI and other rivals are pushing capabilities forward. Coxon's departure asks what happens when researchers inside those laboratories believe the downside is material, admit control is not solved, and still face incentives to keep racing.
As Anthropic moves closer to a potential initial public offering, these internal concerns may weigh on investor sentiment. The company's credit line expansion and reported $2 trillion valuation target suggest significant market optimism, but the resignation highlights unresolved safety questions. For investors, the episode underscores the tension between rapid AI development and the industry's own acknowledged risks.
This article is for informational purposes only and does not constitute financial advice.
