Cybersecurity industry preps for autonomous AI attacks
Add Axios as your preferred source to
see more of our stories on Google.
iCounter CEO John Watters. Photo illustration: Axios Visuals; Photo: Courtesy of iCounter
The day is near when bad actors will use AI to hijack another AI system companies rely on — think chatbots or agents — forcing it to go rogue, according to John Watters, a longtime cybersecurity leader and former leading executive at Google's Mandiant.
- Watters says security companies are now carving out a new vertical of products to respond.
Why it matters: The world is only months away from an untraceable cyberattack run entirely by an autonomous AI agent, Watters said.
- But Watters' twist is even more unsettling: That attack won't be generic. It will be built uniquely for its victim, exploiting a zero-day vulnerability tailored to that company's systems.
The big picture: Security vendors must adapt faster than ever to prepare customers for that new reality. Watters warns that AI tools will make it easier for malicious hackers to personalize their attacks — and to do so at scale.
- That means they won't need to recycle old techniques or turn to flaws in widely used enterprise software to get the most impact. Instead, each strike will exploit a one-of-a-kind weakness in a company with little to no effort.
Reality check: Watters is now CEO of iCounter, a startup building products aimed squarely at that threat, so he's not exactly a neutral observer.
- But he's historically been ahead of the curve. Watters recognized the potential for bug bounty programs decades before they became mainstream, and he had a front-row seat to the evolution of the threat landscape after joining FireEye, which was later acquired by Mandiant.
- After stepping back from cybersecurity in 2022, Watters reentered the field this summer, joining iCounter to help develop its LLM-based tools that spot and block those novel attacks.
Zoom in: Watters predicts that iCounter won't be the only game in town come next spring.
- At next year's RSA Conference — the world's largest gathering of security experts — expect the term AI-DR, or AI detection and response, to dominate the trade show floor, he says.
- Think of AI-DR as a play on the current suite of endpoint detection and response (EDR) tools. But instead of monitoring network endpoints, AI-DR products focus on spotting when adversaries hijack an organization's AI tools — which he says have a huge target on their back because they can be overtaken and forced to hallucinate or go rogue.
- Recent breaches of Salesloft's AI agent underscore the point: Major security companies like Dynatrace, Qualys, CyberArk and Cato Networks are among the latest victims.
What they're saying: "The security gap is the difference between the innovation pace of the adversary and the innovation pace of the defender," Watters says.
- "Adversaries lead. We all think we're innovators — we're not."
Between the lines: Venture capital is betting big on this space, according to a Gartner report released in March. Since 2022, AI-driven detection and response startups have raised more than $730 million.
- Gartner projects that by 2028, 70% of AI implementations in threat detection and incident response will involve multiagent AI — up from 5% as of the report's publication.
Yes, but: Watters expects these tools to drive the conversation at RSAC, but the six months until next year's conference in San Francisco is several lifetimes away in terms of AI development.
Go deeper: Mandiant founder warns of AI-powered cyberattacks
