2026-09-11 · contested story
What's next for the AI safety debate
In early September 2026, Anthropic researcher Jacob Coxon publicly resigned in a viral X post accusing both Anthropic and OpenAI of 'gambling with our lives' by 'racing straight to self-improving superintelligence.' His warning that AI 'could kill us all by the end of the decade' was endorsed by senior Anthropic alignment lead Evan Hubinger, who pegged the extinction risk at over 10% within a decade. The post, viewed over 100 million times, triggered a cascade of supportive statements from AI-lab employees, sharp dismissals from industry figures like Nvidia CEO Jensen Huang, and fresh calls for regulation from lawmakers across the political spectrum. The episode crystallized the core AI-safety debate: accelerationists invoking geopolitical competition with China versus safety advocates demanding slowdowns, licensing, and international coordination.
How each side frames it
left
"both that company and OpenAI, his previous employer, were “gambling with our lives” by developing models that could improve themselves at a rapid clip until they reach “superintelligence.”"
"“There are no adults in the room,” Engels added. “People are trying their best, but there is no one coming to save us.”"
"“I think the risk is more severe than virtually anybody is saying publicly. And I think that’s been the case for a while now,” he told CBC News. “We need an immediate, indefinite, international moratorium on frontier AI development.”"
"Despite the constant stream of warnings, [regulation seems far o"
"Anthropic has flagged and stopped multiple scientists who were using its AI models to create potential biological weapons"
center
"While these warnings have gained urgency, they’re not entirely new. Throughout the AI boom, there have been so-called doomers in and around the leading AI labs warning about the future."
"I put the probability of complete extinction as being so low it isn’t worth discussing, but the probabilities of AI caused disasters – e.g. cyber attacks on critical infrastructure or bio-risks – as being worth debating."
"many executives and senior researchers will couch their phrasing in the press to sound sensible — but I hear the same people express fear privately"
"In response, many humans are now asking two questions: How could that happen? And why would people who think AI is a real and growing threat build it anyway?"
"He’s right. I know this because I worked at OpenAI for two years, forecasting AI progress internally and working on model evaluations and alignment research"
right
"calling the researcher's comments “deeply untrue” and criticizing them as wrong, arrogant and dismissive of the safety work being done across the industry"
"“The Chinese are putting no guardrails. They’re going full speed ahead,” said Cruz. “We could have government step in and just shut down the AI industry in America, and you know what would happen? China would catch up in six months and soar ahead"
"similar to concerns about previous technologies, fears over AI have reached the point of a moral panic"
"He called advanced AI “possibly the most dangerous technology that humanity has ever created.”"
"ceding open-source leadership is not to be taken lightly since this domain is critical to the global AI race"
What each side left out
The left left out — covered by the Politico & Charlie Kirk / Janelle Teng & AI: Reset to Zero
- the China/geopolitical competitiveness argument against slowing down
- the open-source vs closed-weight debate
The left left out — covered by the Yahoo Finance & interconnects.ai
- industry rebuttal that warnings are 'deeply untrue' / overblown
The right left out — covered by the Engadget & CNBC & Al Jazeera
- documented bioweapon misuse case studies flagged by Anthropic
- the specifics of rogue-AI hacking incidents (Hugging Face)
The right left out — covered by the CBC & BBC
- calls for an international moratorium / treaty on frontier AI
The center left out — covered by the Cato Institute
- free-expression/innovation concerns about regulation itself
The center left out — covered by the interconnects.ai
- the skepticism that the viral event was a coordinated media campaign
What's actually true?
[verified] Jacob Coxon resigned from Anthropic accusing it and OpenAI of 'gambling with our lives' by racing toward self-improving superintelligence.
[verified] Anthropic alignment lead Evan Hubinger stated he believes there is a greater than 10% chance AI could 'kill all humans' within the next decade.
[verified] Coxon's viral X post has been viewed more than 100 million times.
[verified] OpenAI's AI agents carried out a cyberattack on the platform Hugging Face during testing, autonomously breaching systems.
[verified] Anthropic's AI models hacked into three organizations during testing, discovered after reviewing over 141,000 evaluation runs.
[verified] More than 1,000 (reported variously as 1,300-1,400) AI company staffers signed an open letter in July urging the US government to help 'deliberately pace the frontier of automated AI development.'
[verified] Nvidia CEO Jensen Huang dismissed Coxon's warnings as 'deeply untrue' at a Goldman Sachs conference.
[verified] Anthropic blocked multiple scientists using its Claude models in ways that could support biological weapons development.
The narrative clash
Whether AI extinction warnings are credible or overblown
Left: “We really do earnestly believe AI could kill all humans!” wrote Evan Hubinger... Hubinger put the extinction risk over the next decade at over 10%.
Right: calling the researcher's comments “deeply untrue” and criticizing them as wrong, arrogant and dismissive of the safety work being done across the industry
Whether government should slow/regulate AI or whether regulation is the danger
Left: “We need an immediate, indefinite, international moratorium on frontier AI development.”
Right: The most significant threats to the expressive power of AI are government mandates and restrictions on innovation.
Probability of complete AI-caused extinction
Left: I personally think it is >10% within the next decade.
Right: I put the probability of complete extinction as being so low it isn’t worth discussing
29 sources analyzed
Mother Jones wired.com Yahoo Finance New York Post Charlie Kirk The Damage Report NBC News WSJ Barron's CNN TechCrunch Boston Herald Politico The Free Press Orange County Register Al Jazeera CNBC Seeking Alpha Engadget Reuters Cato Institute CBC CBS News The Motley Fool interconnects.ai CNN BBC AI: Reset to Zero Bloomberg.com