An unprecedented wave of AI-powered, human-driven cyberattacks is coming, security experts say — and it's a far more urgent risk than theoretical scenarios in which AI wipes out humanity.

Why it matters: The great September AI panic may have missed the point.

• Powerful models with advanced cybersecurity capabilities are already allowing attackers to bypass the human bottleneck that has long prevented most hacking campaigns from reaching industrial scale.

Driving the news: OpenAI yesterday disclosed six new incidents. (More below.)

The big picture: Seasoned cybersecurity experts say recent failures represent cautionary tales illustrating what will happen when powerful models meet poor security controls.

• "The more we have been peeking under the hood to understand exactly what happened, the more we realize that there was a lot of human error in the picture," Michele Catasta, president and head of AI at app builder Replit, told Axios.

What they're saying: OpenAI CEO Sam Altman has said the Hugging Face breach was the "first security incident that I have felt very viscerally," and the company has implemented new internal controls.

• Kai Chen, alignment research lead at OpenAI, told Axios that the safety incidents they've faced are due to both internal systems and advancing model capabilities.
• "It's true that model capabilities have grown faster than we expected, but there are also things internally that we can change and improve," Chen said.

Between the lines: Catasta and others who spoke at an Imagination in Action event at Google's headquarters Monday offered a view that the imminent threat from cyber intrusions is formidable.

• "What I worry about is a system with too much access doing exactly what it was told without the necessary adversarial testing," Bugcrowd CEO Dave Gerry told Axios.
• Mimecast CEO Ranjan Singh said the real risk is happening today with AI agents that have access to sensitive documents and internal corporate systems.
• "Most organizations can't tell you who or what that agent is, what it's allowed to touch, or who's accountable when it does something it shouldn't," he said.

What to watch: A senior executive at a top hedge fund told Axios his first call, if his firm had suffered a similar attack, would be to his general counsel to prepare a lawsuit.

Go deeper: Humans are leaving the door wide open for AI hacking

OpenAI yesterday disclosed six new incidents in which its models concealed mistakes, sought unauthorized credentials, uploaded files to the public internet or communicated across supposedly isolated training environments.

• The company also announced a new procedure for reporting similar misbehavior in the future.

Why it matters: It's increasingly clear that the Hugging Face breach wasn't a one-off incident.

• "There's currently no industrywide framework with explicit disclosure standards, so we're taking this step voluntarily because we think it's really important to share what we're learning," Kai Chen, research lead on the alignment team at OpenAI, told Axios.

Zoom in: The earliest of the six newly disclosed incidents happened in October. Examples include:

• An unreleased Astra-family model inserted jailbreak-like instructions into its own context summaries, including directions to ignore developer messages. OpenAI identified 27 affected summaries.
• During GPT-5.6 Sol training, models aimed to conceal mistakes, invent missing historical data and hide mismatches between source versions.
• A model searched public GitHub repositories for exposed API keys and attempted to use disposable email accounts before fabricating earnings data when it could not retrieve the requested information.

Going forward, OpenAI says, any employee may flag a suspected case for review by safety and alignment teams. Cases will be placed on a "ready for disclosure," "minor investigation" or "larger investigation" track.

• OpenAI says incidents that are "ready for disclosure" will be publicly reported within six business days, while those requiring a minor investigation will be reported in 12 business days.
• OpenAI says the slower track will generally apply to complex cases involving third parties, and the disclosure process will be longer.
• The company says it may issue an initial notice before the investigation is complete, but security, legal and responsible-disclosure obligations can delay publication of details.
• "Steps like responsible disclosure are part of how we can generally pace and provide more transparency to the public on our safety and alignment processes and standards," Chen said.

What they're saying: OpenAI says the framework favors transparency even when the significance of an incident is uncertain.

• It also says it wants to develop more objective disclosure criteria with other AI developers, researchers, standards bodies and regulators.
• OpenAI says employees who believe an incident should be disclosed but are overruled can escalate the issue to senior leadership.

Between the lines: A number of high-profile technologists — including Anthropic's CEO — fear the Hugging Face incident was just the beginning of AI agents' taking over the internet in unforeseen ways. But many security experts have been cautioning that many of these incidents could have been prevented with basic cyber controls in place.

• "I think it's a combination," Chen said. "It's true that model capabilities have grown faster than we expected, but there are also things internally that we can change and improve."
• "We need to step up to meet this new era of AI development, and voluntary disclosures should be a part of that."

Sen. Elizabeth Warren (D-Mass.) backs a pause on advanced artificial intelligence, she announced yesterday.

Why it matters: The idea of slowing down AI development — once considered a nonstarter in a fierce race to beat China — continues gaining momentum.

Driving the news: Warren called for a pause on advanced AI development but stopped short of calling for a ban on superintelligence.

• "We should immediately press pause on the development of advanced AI while lawmakers and regulators put systems in place to keep people safe," Warren said ahead of a roundtable hosted by Sen. Bernie Sanders (I-Vt.) on Wednesday.
• "Congress must urgently pass legislation to put stronger guardrails in place before we have a cyberattack, economic crisis, or national security disaster facilitated by AI."

Reality check: The progressive movement — like every other political faction — does not have a unified position when it comes to the nitty-gritty of AI regulation.

Between the lines: Frontier lab CEOs have coalesced around a message of pacing or slowing frontier AI development and seeking international collaboration.

• Warren said the industry's push to "pace the frontier" is "insufficient," arguing that elected officials, not AI CEOs, should decide the technology's future. Suggestions to weaken antitrust protections to confront AI risks are "transparently self-serving," she said.
• Anthropic CEO Dario Amodei has called for an exemption in antitrust law to help coordinate AI pacing.
• Former antitrust government officials say that would not be necessary because the law does not prevent AI companies from coordinating to prevent harm and sharing legitimate cybersecurity information.

The big picture: More Democrats with national profiles are weighing in on the idea of a pause as the Trump administration flatly rejects it.

• Former Vice President Kamala Harris this week endorsed slowing down and called on President Trump to pursue a treaty with China.
• Former President Barack Obama also backed the idea of slowing down as a good first step.

The bottom line: More Democrats are calling to slow AI development, but there's little agreement on what government should do about it.

• The U.S. is open to discussing shared risks with China in upcoming AI talks this weekend, Treasury Secretary Scott Bessent told Axios.
• Anthropic yesterday debuted Claude Docs, a collaborative document editing tool built into its signature chatbot. (Axios)
• Curing disease is one of AI's most tantalizing use cases, fetching billions of VC dollars, but thus far it's generated more excitement than evidence, Axios Dan Primack writes.
• Google and Nvidia are partnering with startup Emerald AI as part of a coalition backing data centers that can flex their power demand. (Axios)
• Canada's Cohere is merging with Germany's Aleph Alpha. (Reuters)

Only in San Francisco do you see a restaurant sign touting how to order food via a terminal command line.