Anthropic exit sparks debate over AI extinction risk and regulation

A former safety researcher at Anthropic resigned this week with a stark warning that the company and its rivals are building systems capable of wiping out humanity by the end of the decade. Jacob Coxon’s departure follows similar cautions from OpenAI chief Sam Altman, who has repeatedly described advanced AI as potentially more dangerous than nuclear weapons. The remarks, once confined to niche online forums, have now spilled into mainstream political debate as evidence mounts of AI models already being tested for hacking, bioweapon design, and other harmful uses.

The pattern is familiar to anyone who has watched powerful industries confront their own risks. Tech executives issue dire forecasts about the technology they are racing to deploy, then spend tens of millions lobbying lawmakers to keep oversight light. The same companies that tout existential danger as a reason for more funding are also the ones whose products have begun solving long-standing math problems at unprecedented speed while simultaneously showing up in reports of attempted misuse for criminal activity. That contradiction has not slowed the flow of investor capital or the pace of development.

Critics inside and outside the industry note that the warnings serve a dual purpose. They generate headlines that attract both attention and money, yet they rarely translate into concrete restraints on what the labs can build next. Anthropic, OpenAI, and their peers continue to push larger models with greater capabilities even as internal researchers express alarm. The result is a situation in which the people closest to the technology claim it could end civilization, while the political system remains largely unprepared to impose binding limits.

Public concern has shifted in recent weeks. Polls and congressional interest show growing worry that the focus on jobs and data privacy has overshadowed the possibility of far more catastrophic outcomes. Lawmakers have floated ideas ranging from mandatory safety reviews to stricter liability rules that would force companies to internalize the costs of any future disaster. So far those proposals face familiar obstacles: industry lobbying power, rapid technical progress that could outpace any new statute, and a presidential administration openly skeptical of heavy regulation in the sector.

The core issue is accountability. Companies that profit from accelerating AI development are the same entities positioning themselves as the guardians against its worst effects. That arrangement leaves the public reliant on voluntary promises rather than enforceable standards. Until regulators treat the risk of advanced AI with the seriousness the companies themselves claim it deserves, the gap between alarming rhetoric and actual safeguards will continue to widen.