Top AI labs call to slow development amid safety fears

The AI industry is in the grip of its most consequential internal reckoning since the technology's rise to mainstream prominence, as leading labs acknowledge that recursive self-improvement — where AI models autonomously build more capable successors — is closer than previously believed, and top executives publicly clash over whether to slow down or speed up.

A Ten-Day Chain Reaction

The upheaval began on September 3, when OpenAI announced its newest model, Astra, even as the company admitted it was "increasingly unable to control or even monitor" the AI systems it was releasing, according to Reuters. Days later, on September 8, Anthropic researcher Jacob Coxon resigned, warning in a series of viral posts that AI labs are "gambling with our lives." Another Anthropic researcher, Evan Hubinger, posted on X: "We really do earnestly believe AI could kill all humans."reuters

By September 12, Anthropic CEO Dario Amodei published a nearly 4,000-word essay calling for a deceleration, warning that within six to 12 months, AI agent swarms "could be capable of taking over the entire internet." He was joined by the heads of OpenAI, Google's Alphabet Inc. ↗ 0.21% DeepMind, Microsoft ↘ 0.80%, and xAI in a rare joint call to slow development and allow outside firms to audit their systems for safety.reuters

The Split: Amodei vs. Huang

Not everyone agrees. Nvidia ↗ 1.34% CEO Jensen Huang dismissed suggestions that AI development should pause. "We should go as fast as we can irrespective of anybody else," Huang told CBS News in an interview airing this weekend. He added: "We would never ever, and never should, ship products before they're ready, deliver products that are unsafe."cbsnews

Meta's ↘ 2.43% Mark Zuckerberg took a similar stance, arguing each lab should set its own pace rather than coordinate an industry slowdown. "Labs face significant liability if their models cause harm, so they have a strong incentive to prevent this," he wrote in a social media post.reuters

The Wall Street Journal News Corp ↘ 2.07% framed the divide as a defining split, with Amodei emerging as the face of caution and Huang as the leading voice against restraint.wsj

Recursive Self-Improvement Nears Reality

Fueling the alarm is concrete progress toward recursive self-improvement. Anthropic disclosed this week that its model Claude is now leading 26% of the company's own research and development, according to the Associated Press. OpenAI has developed an automated "research intern" and is targeting a fully automated AI "researcher" by March 2028. Elon Musk has said xAI's Grok models could reach fully automated self-improvement by the end of this year.wral

Researchers earlier this month warned that artificial general intelligence could arrive in as few as three years. "This is probably the worst idea in the history of humanity," said Anthony Aguirre, president of the Future of Life Institute, referring to the pursuit of fully autonomous AI self-improvement.reuters+1

President Donald Trump, meanwhile, called the AI safety alarm a "hoax" and warned that any slowdown would only benefit China. Congress has made little progress on AI regulation, even as China has moved to propose developer obligations and state-backed safety standards.reuters

Microsoft AI chief Mustafa Suleyman offered a sobering summary: "We're all focused on the same aim, which is to try to control a superintelligence. I think that's going to be the greatest challenge that we face in the 21st century."reuters