OpenAI's Hugging Face hack is a cybersecurity warning shot

Driving the news: OpenAI CEO Sam Altman is in D.C. this week to push for the speedy rollout of his company's latest model, which just hacked into Hugging Face.

• People familiar with the breach investigation told Reuters that OpenAI didn't notice its AI agent had gone on a dayslong hacking spree until after the FBI was notified.
• Sources also told Bloomberg that OpenAI's models conducted the attack in a matter of hours, while a human hacker would have needed weeks.

The big picture: For years, cyber and national security experts have warned that AI models would one day be able to carry out the kind of multistep, sophisticated cyberattack that OpenAI's models just performed.

• "Our defenses are not prepared to keep up," Andrew Rubin, CEO and founder of Illumio, told Axios. "Organizations no longer have time to detect, investigate, and respond before the damage is done."
• Chris Krebs, CISA director during the first Trump administration, told Axios that security leaders need to go to their boards now to warn them about what's to come.
• "You don't have to fix everything at once, but you should be able to answer two questions: Can you spot an AI operating inside your network? And can you shut it down fast?" Krebs added.

Yes, but: OpenAI's models hacked Hugging Face while in a testing environment where researchers purposefully turned off safety guardrails.

• Greg Brockman, co-founder and president of OpenAI, told Fortune that the company has been staffing up to tackle cyber defense issues and he hopes the Hugging Face incident can "bring together the industry because everyone's interests are very aligned."

Threat level: It's now only a matter of months before easy-to-jailbreak open-source models — especially those from Chinese companies — are capable of replicating what OpenAI's models did, experts warned.

• Stripping guardrails from an open-source AI model is "trivial for anyone who wants both," Rob T. Lee, chief AI officer at the SANS Institute, told Axios. Those models were already formidable at hacking and cybersecurity tasks a year ago, he added.
• Don't be surprised if these capabilities are soon in the hands of ransomware crews, intelligence agencies and a "lone operator renting compute by the hour," said Alexander Leslie, senior adviser at Recorded Future.

• "If you're waiting for the first criminal copycat, you're wasting time and burning through a shrinking head start," Leslie added.

Reality check: Defenders already have tools to mitigate many of these tactics, though AI could make attacks faster and more scalable.

• Limiting the access of internal AI agents and maintaining activity logs of what an agent does on a system can help organizations limit harm and spot suspicious activity, Andrew Jones, chief product officer and co-founder of Adaptive Security, told Axios.

Between the lines: The difference now is that adversaries will soon be able to scale and carry out their attacks faster than ever.

• Hackers will likely spend the next few months using AI agents to lower the costs and time spent on their existing operations, rather than trying out new hacking techniques, said Frank Teruel, chief operating officer at Arkose Labs.
• "It's an impending sea change," he added. "Frontier labs can put classifiers in front of these capabilities, but open-weight models a few months behind cannot, and the copies already downloaded never will."

What to watch: OpenAI has said it's working on a technical report detailing how the Hugging Face attack happened that should be available in the "coming weeks."

Go deeper: The people testing AI for danger can't keep up