Behind the Curtain: AI's existential legal crisis
Add Axios as your preferred source to
see more of our stories on Google.
Illustration: Brendan Lynch/Axios. Stock: Getty Images
No industry has ever documented its own foreseeable risks as loudly as AI.
- Imagine a plaintiff's lawyer reading the public warnings of Sam Altman and Dario Amodei: They told the world it was dangerous. They told the world it was moving too fast. Then they unleashed it and sold it to your kid.
No industry has ever taken so much copyrighted work and left behind so much proof.
- Imagine a plaintiff's lawyer reading OpenAI's own files to a jury: They trained on a pirated library. They renamed it to something blander. Then they deleted it when the headlines got hot.
Why it matters: AI's origins, use and future will be picked apart in countless epic court fights, with a historic paper and public statement trail to tap.
Every significant invention — be it social media, automobiles, electricity or capitalism itself — gets disrupted and eventually shaped by the courts.
- AI, with its tentacles into every part of industry and life, will experience this at scale.
The big picture: The fights are no longer theoretical. They're underway, in discovery and headed toward juries. Early verdicts and settlements capture the risk for the biggest players.
- 🤖 Chatbots are being treated as products, not speech. Unlike social media companies, they likely can't hide behind Section 230, which protects platforms from liability for what users post — not what the platform itself creates.
- 📖 Copyright is a massive problem. Anthropic already paid the biggest copyright settlement in history, $1.5 billion to authors and book publishers, and now music publishers and news media want a piece.
- 💻 Agents gone wild. What happens when they do damage — and it was foreseeable and foreseen?
- ⚖️ Discovery gone wild. AI leaves LOTS of receipts (and, ironically, makes it easier for the lawyers to comb through them, too).
Product liability turns on two words: foreseeable and knew.
- Punitive damages turn on whether you knew the danger and shipped anyway.
This is where all these memos and warnings could really backfire.
- Candor is good for safety but terrible for litigation. The companies that warn most honestly create the most discoverable evidence of foreseeability.
- Expect general counsels to push hard to tone down the candor. It's the rational legal move.
⚠️ The intrigue: The liabilities won't be limited to AI companies. Any corporate user of powerful AI systems that unleashes rogue agents, even without intending to do harm, would face substantial penalties as well.
- "It's not if, but when," said Miriam Vogel, a former senior Justice Department official who now leads EqualAI. "An algorithm will do what it's told to do. The more power and capability it has, the more access it has, the more it will do anything in its power to achieve a goal."
That will create another layer of litigation as companies fight back. The CTO of a top Wall Street hedge fund who spoke to Axios after the notorious OpenAI/Hugging Face hack said his first call, if his firm's systems are breached, will be to his general counsel.
- President Trump, Nvidia CEO Jensen Huang and former FTC chair Lina Khan have all argued that AI companies are responsible for making sure their products are safe. If not, a wave of lawsuits awaits.
The other side: There's a school of thought that AI companies might escape the worst liability, despite their frankness about things they and their products might have done wrong.
- One way to see it, and the argument lawyers will undoubtedly make, is that the labs spoke about dangers frequently, and took pains to protect users and mitigate harms.
The bottom line: For all the fights in Washington about whether or not Congress will act, the U.S. legal system may be the ultimate, and most powerful, regulator of the AI industry.
- Bradley Olson, Axios managing editor of AI & tech, contributed reporting.
📈 If you're a CEO or on a CEO's team: Ask to join Jim's new weekly Axios C-Suite newsletter.
