First look: New warning calls for slowing race to superintelligence
Add Axios as your preferred source to
see more of our stories on Google.
Illustration: Aïda Amer/Axios
The group behind a 2025 report predicting dire outcomes from AI development is out with a new prescription: to avoiding dangerous outcomes from superintelligent AI, slow everything down.
Why it matters: The researchers behind the new essay, first shared with Axios, are saying that while people can't stave off the development of superintelligence, they can slow it down so the world is ready when it comes.
Driving the news: The proposal by the AI Futures Project, a nonprofit AI research foundation, calls for delaying the development of superintelligence to 2040.
- They'd do that by slowing the research timeline, forcing more transparency from AI labs and spreading power across more companies and countries.
- The group's first essay, AI 2027, was read by Vice President JD Vance and created waves among AI researchers and the media.
- The six authors of the new essay include Daniel Kokotajlo, a former OpenAI researcher, and Thomas Larsen, who co-wrote AI 2027.
What they're saying: "If you delay the advent of superintelligence, then that gives society more time to prepare and more time to solve the various problems that it represents," Kokotajlo said in an interview with Axios.
- "We're currently on track for this really scary status quo," Larsen said. The plan calls for "carefully reasoned interventions" so the development would happen "over the course of many, many years, maybe a decade or so, as opposed to just a single year."
How it works: Safety problems — like losing control of AI, governments having inadequate time to grapple with policy, concentrated AI power in countries and companies and massive job loss — can be blunted by slowing down the timeline, the essay argues.
The big picture: "We recommend an international deal between all major world powers to avoid a dangerous race to superintelligence," the report's authors write, and the U.S. and China should agree to a "verified slowdown."
- That would involve "multiple companies across multiple countries scaling slowly and safely towards superintelligence instead of racing each other in secrecy," per the essay.
- AI companies should be transparent about "everything but the model weights," Kokotajlo said, so outside groups can "check the AI company's homework."
The bottom line: The essay's authors know they are facing a tough geopolitical reality, with a gridlocked Congress, a Trump administration that just came around to the idea of AI safety, global competition and tensions with China.
- "We think it's still good to recommend what would actually be good, even if you think that your audience is probably not going to listen," Kokotajlo said.
