These 5 AI risks have the highest potential for catastrophe
Add Axios as your preferred source to
see more of our stories on Google.
There is a one-in-five chance of AI gaining dangerous weapons capabilities or causing mass harm that could kill millions in the next five years, per global experts surveyed for a recent MIT study.
Why it matters: The findings add to the growing debate over AI safety and cybersecurity as governments and companies race to deploy increasingly capable systems.
- Last week's news of an OpenAI model breaking containment and breaching the AI platform Hugging Face only adds to the debate about what can go wrong.
Driving the news: Researchers from MIT and the University of Queensland in Australia asked 272 international experts to evaluate 24 different risks based on their likelihood and severity between 2025 and 2030.
- The experts said 18 of the 24 AI risks had "at least a 10% probability of catastrophic outcomes over the next five years" — but five main risks stood out with higher probabilities.
- The study defined a "catastrophic outcome" as an event having more than 1 million human deaths, more than $100 billion in financial loss or generally "civilizational-scale intangible impacts."
The five catastrophic risks if companies and organizations stay on the current course include:
- AI possessing dangerous capabilities: Experts say these could potentially cause mass harm through deception, weapons development, cyber offenses and misalignment (21.5%).
- Cyberattacks, weapon development or use, and mass harm: Using AI to develop weapons (including chemical or biological), commit cyberattacks or create mass-harm weapons, like nuclear explosives (21%).
- Power centralization and unfair distribution of benefits: Limiting which people or groups benefit from AI could cause societal inequality (18%).
- Competitive dynamics: AI competition to release the best models leads to error-prone systems (16.6%).
- False or misleading information: AI creates misinformation or errors that lead to inaccurate beliefs, leading humans to make poor decisions based on false beliefs (12.8%).
Reality check: The experts said some mitigations could reduce the severity of the AI harms, but the chances of them happening remained above 10% in that scenario.
The big picture: AI safety concerns have continually raised questions about guardrails.
- AI behaviors, as well as the opinions about those behaviors, can move markets and reshape conversations overnight — leading to doomsday scenarios.
The other side: AI has driven a productivity explosion, with many in the AI field cheering the new technology's ability to improve productivity and automate tasks.
The bottom line: Researchers say catastrophic AI risks remain uncertain but significant enough that interested parties should treat them as challenges to overcome.
Go deeper: AI is a cybersecurity risk accelerant, experts say
