The old ways of testing and evaluating new frontier AI models need a rewrite.

Why it matters: AI models are outgrowing the existing methods of testing and benchmarking their hacking abilities — and without new tests, policymakers and corporate security teams won't have a clear way to predict what these models can actually do or whether they can be deployed safely.

Driving the news: Federal agencies have until Aug. 1 to establish a classified benchmarking process to assess the capabilities of frontier AI models, although the Financial Times reports those standards may arrive as soon as this week.

• When Fable 5 returned last week, Anthropic said in a blog post it was creating a standardized benchmark with Amazon, Google, Microsoft and other partners that focuses on the outcomes and impact of a jailbreak, rather than simply whether one is possible.

The big picture: Even before the government began rethinking how to evaluate frontier AI models, industry was already redesigning the way their cyber capabilities are measured.

• Irregular — a testing lab that works closely with frontier AI labs including OpenAI and Anthropic, as well as governments — released a new cyber benchmark in late June that measures whether AI models can carry out offensive cyber tasks such as remote code execution, privilege escalation and reaching a restricted network.
• Other groups and companies, including Wiz and Vals AI, have been developing benchmarks that measure how well AI models perform similar offensive cyber operations.
• Stanford warned in its 2026 AI Index that "evaluations intended to be challenging for years are saturated in months."

Between the lines: While each effort measures something different, they all reflect the same shift: static tests no longer capture how frontier AI systems behave in realistic environments.

• Earlier benchmarks focused on isolated tasks, such as solving a predictable, staged hacking challenge or discovering previously fixed vulnerabilities that weren't included in a model's training data.
• But the agentic and reasoning capabilities of advanced AI models like Mythos Preview and GPT-5.5 are rapidly outpacing those tests, making it harder to understand what these cyber-capable systems can actually accomplish in practice.
• "We're testing maybe the most bare bones fundamentals of capabilities," David Slater, co-founder of AI red-teaming company Armadin, told Axios. "We are very far away from measuring whether this thing can, in a real environment, do something dangerous."

Zoom in: Slater said his company's AI agents surpassed every public cyber benchmark within four weeks, using a combination of additional training and human expertise.

• By the last quarter of 2025, the company — which offers continuous AI red teaming — had concluded that public cybersecurity benchmarks were "totally saturated" and "useless," he added.
• The next generation of benchmarks needs to measure whether models can carry out longer, more sophisticated cyberattacks and how much effort or cost is required to do so, he said.
• That includes testing models in environments that resemble real production systems, providing a better indication of how quickly they can bypass security controls or move laterally through a network.

Yes, but: AI models are also getting better at attempting to break out of sandboxed environments, making it harder for defenders to evaluate them in isolated settings that don't interact with production systems, Slater said.

• "The jailbreak attempts are nuts," he said. "We see this thing trying to escape and get out onto the cloud container that it's running on, using keys that it has access to, to do crazy stuff."

What to watch: All eyes are on how Washington decides to evaluate the cyber capabilities of U.S. frontier AI models — especially as leading AI labs push back on the current, largely ad hoc testing process.

A recently patched flaw in a popular Google Cloud service that powers AI chatbots could've allowed attackers to hijack customer conversations and trick users into handing over sensitive information, according to a Varonis report shared first with Axios.

Why it matters: Companies are increasingly relying on AI chatbots to handle customer service, health care and financial interactions, making flaws in these systems rich targets for attackers.

Driving the news: Varonis found a critical vulnerability in Google's Dialogflow CX platform, which companies use to build AI-powered customer service chatbots and voice assistants.

• The service is widely used to power customer support chats, financial services bots and health care assistants.
• Varonis researchers found that someone who compromised one chatbot could silently monitor conversations, impersonate the bot and, in some cases, interfere with other AI chatbots running in the same Google Cloud project.

Threat level: Users could have been tricked into sharing passwords, insurance information or financial data that attackers could then use in future cyberattacks, Matthew Radolec, field CTO at Varonis, told Axios.

• Varonis discovered the issue in November. Google issued an initial security update in April and fully resolved the issue last month.

Yes, but: Varonis said it found no evidence the vulnerability had been exploited in the wild before it was patched.

• "We appreciate the efforts of researchers like Varonis who disclose their findings through our Vulnerability Reward Program," a Google Cloud spokesperson told Axios in a statement. "The underlying issue has been fully mitigated, and we have no known indication of customer compromise. No customer action is required."

Between the lines: Radolec argues AI tools are being adopted faster than technology companies can fully secure them.

• "This whole concept of 'zero trust' architecture is supposed to be leading the charge in cloud and AI, and this is a case where that was overlooked," he said.

The bottom line: As companies rush to deploy AI, security teams should verify that AI tools are properly isolated and should routinely check for exposed credentials.

The world's largest AI companies have weakened key safety commitments even as their models grow more powerful, according to a new report from the Future of Life Institute.

Why it matters: The report suggests that the voluntary safety system created by AI labs has begun eroding before governments have put a durable alternative in place.

Between the lines: Anthropic ranked first in the institute's latest AI Safety Index, but it received only a C+ overall, with OpenAI and Google DeepMind each receiving a C.

• Meta improved to fourth from sixth, while xAI fell to seventh from fourth.
• xAI, DeepSeek and Mistral received failing overall grades — one company each from the U.S., China and Europe.

Zoom in: The reviewers said Anthropic, OpenAI, Google DeepMind and Meta have weakened or eliminated earlier commitments to pause development if their systems approach specified danger thresholds.

• The panel described the changes as "moving the goalposts" and said they have undermined safety frameworks across the industry.
• "AI companies are sprinting toward a cliff," FLI chair Max Tegmark said in the institute's release. "Despite acknowledging the great risks of artificial superintelligence, they continue racing to build it."

The big picture: The report comes after experts sounded similar warnings at a UN conference on AI in Geneva yesterday.

• "We may be the last generation able to set the terms on which humanity and machines coexist," UN Secretary-General António Guterres said at the gathering, which included a number of dire predictions for AI's future unless global governance is quickly strengthened.

Read the rest.

@ D.C.

🤝 An inside look at how the Trump administration and Anthropic resolved a 20-day showdown that led to the shutdown of Fable 5 and Mythos 5. (Axios)

📋 Sen. Mark Warner, ranking member of the Senate Intelligence Committee, is working on a bill that would create an FTC registry certifying the privacy and security protections for agentic AI software. (CyberScoop)

🗂️ Hackers recently breached a Department of Homeland Security database used to share data among federal, state, local and industry partners. (Nextgov)

@ Industry

👀 Chinese authorities have held meetings with top tech firms about potentially restricting overseas access to the country's top AI models. (Reuters)

🔍 After backlash from privacy advocates, Anthropic removed code that it had quietly deployed on Claude Code to track the activities of China-based customers. (Washington Post)

🛡️ Cyber insurers are shifting the qualifications and requirements for obtaining policies as AI starts to change the threat landscape. (Wall Street Journal)

@ Hackers and hacks

🤖 Researchers have uncovered what they believe is the first documented case of an agentic ransomware operation in the wild. (CyberScoop)

🚔 A 19-year-old affiliate of the prolific Scattered Spider cybercriminal group was arrested in Finland and extradited to the United States. (BBC)

🎟️ Anthropic's Claude Opus 4.7 helped a hacker break into a popular music festival ticketing website and issue himself and his friends free VIP backstage passes to Bonnaroo. (Wired)

Even San Francisco's Karl the Fog showed up to watch the rare fireworks show off the Golden Gate Bridge this year!