Anthropic safety researchers quit, warn AI could threaten humanity

Joe Benton, a former manager of Anthropic's Scalable Oversight team, publicly announced on September 11 that he had left the company two weeks earlier, warning that AI firms are racing to build systems "much smarter than any human" without investing enough in safety. His departure marks the latest in a string of resignations from leading AI labs by researchers who say the technology is advancing faster than the safeguards meant to contain it.blockonomi+1

Benton joins former Anthropic pretraining researcher Jacob Coxon, who resigned on September 8 and accused both Anthropic and OpenAI of "racing straight to self-improving superintelligence and gambling with our lives". A Google DeepMind researcher, Josh Engels, also left around the same time, with both Benton and Engels joining METR, an independent nonprofit that evaluates frontier AI systems.apnews+3

A Growing Chorus of Alarm

Benton warned that humanity "may not survive" the transition to machines far more intelligent than humans. He said AI capabilities are "already improving extremely fast, and the companies are trying to go even faster". He predicted a future in which humans could share their world with AI agents much smarter than themselves within just a few years.newsbytesapp+1

Evan Hubinger, Anthropic's Alignment Science Lead, amplified the warnings by writing on X that he "earnestly believe[s] AI could kill all humans," pegging his personal estimate at greater than 10% within the next decade.forbes+2

Calls for Transparency

Benton's central concern extends beyond job displacement to the emergence of advanced systems with capabilities and goals beyond human control. He pointed to a July cyberattack on Hugging Face involving autonomous AI systems as a warning, saying such incidents highlight "the growing gap between what AI systems are told to do and what they may actually decide to do".freepressjournal+1

"At the minute, basically all of the transparency about these risks that is coming from the companies is entirely voluntary," Benton told NBC News. He called for independent assessments of AI safety measures and greater public disclosure about capability improvements, safety incidents, and near-misses.nbcnews+1

Political Pressure Mounts

The warnings have drawn attention from U.S. lawmakers. According to the Straits Times, both Democrats and Republicans, including key senators, have called for new oversight rules requiring AI companies to work with government experts to verify and test models. No federal law currently mandates that companies like OpenAI and Anthropic share reports when AI systems act beyond human control.straitstimes+1

Anthropic has acknowledged that powerful AI brings risks and has highlighted its safety research, but reports indicate the company did not share its newest model with the UK AI Security Institute for risk testing.technobaboy