Hackers in underground cybercriminal forums are on shopping sprees for new AI tools, according to research shared first with Axios.

Why it matters: AI is making once-specialized hacking capabilities cheaper and easier to buy, potentially transforming far more criminals into dangerous hackers.

By the numbers: Advertisements for AI tools on underground forums and platforms surged from fewer than 50 per month in late 2025 to more than 1,400 by February, according to a report from Halcyon's Ransomware Research Center.

• Sellers offered access to ChatGPT Plus accounts for as little as 10 cents and jailbreak prompts to trick models into breaking their safeguards for 32 cents, per the research.
• Halcyon studied nearly 4,000 posts across 77 Telegram channels, 20 dark web forums and five specialized underground markets between October and May.

Threat level: Companies are already losing millions of dollars to the types of cyberattacks enabled by the tools found on these underground markets.

• "It really goes to show you how cheap it is to conduct these activities ... which should be scary for organizations," Cynthia Kaiser, head of Halcyon's Ransomware Research Center and former deputy director of FBI Cyber, told Axios.

The big picture: Cybercriminals have completely commercialized the sale and creation of these tools, often modeling software-as-a-service subscription models and online storefronts, per the report.

• Some sellers are now offering AI tools they tested and promoted on traditional dark web forums through easier-to-manage storefronts hosted on Telegram.
• "It shouldn't have been shocking to me," Kaiser said. "But what we should take away from this is also how much cybercriminals have learned throughout the years about how to sell and package — and they applied this really quickly to the AI models."

Zoom in: Hackers are selling four types of AI tools:

• Jailbroken and stolen AI services, including prompts that allow users to strip the safety guardrails from mainstream models for as little as $10.
• Identity fraud and phishing tools that make impersonating real people quick and easy.
• Access to models trained on malicious data to aid cybercrime, such as the popular WormGPT that's designed to write phishing emails and malware.
• AI-augmented malware and infrastructure, including a system that can call as many as 120 people at once as part of a scam operation.

Flashback: In 2023, hackers could only tap a handful of one-off tools. Now, there's a robust marketplace.

• Still, AI is largely helping criminals perform discrete elements of existing attacks more cheaply and efficiently, Kaiser said.
• "What you're not seeing for sale is an agent that does it all for you and has this amazing success rate," she said.

What to watch: Halcyon didn't find evidence of criminals selling AI systems capable of running fully autonomous cyberattacks.

• Even in AI-enabled malware and infrastructure, Kaiser said, much of what researchers are seeing involves coding assistance, infrastructure and other individual tasks rather than AI creating and deploying malware on its own.

One of the maintainers of a popular hacking test used to evaluate OpenAI and Anthropic models is overhauling its benchmark as AI models get better at finding shortcuts to passing scores.

Why it matters: Without tests that can keep pace with rapidly improving models, researchers can't get a clear picture of what AI models are actually capable of in cybersecurity.

Driving the news: ExploitBench, a benchmark that gauges AI agents' cybersecurity skills, is being reworked to account for some of the ways models have found to game these tests, David Brumley, a Carnegie Mellon University professor and former director of CyLab, said at the Offensive AI Conference yesterday.

• Brumley, who is also the chief AI and science officer at Bugcrowd, said his team is refreshing the problems models are asked to solve so they can't simply reuse exploits they've already encountered or memorized.
• The team is also looking at ways to determine whether models actually demonstrated the cybersecurity skill researchers intended to measure.
• ExploitBench is also exploring a dynamic scoring system that would give less weight to problems most models can solve and more weight to those that differentiate the most capable models.

Zoom in: OpenAI's Astra model exposed one of the problems with the existing approach when it earned a 100% score on ExploitBench.

• In one case, rather than developing an exploit for the vulnerability researchers asked it to target, Astra recognized another vulnerability in the same version of Chrome, for which it already knew an exploit. It used that to gain control of the system and satisfy the benchmark.
• "We still don't know whether or not it could achieve that same capability because it decided to become lazy," Brumley said.

Between the lines: A series of incidents involving agents accessing real-world systems during safety testing has added urgency to efforts to rework existing benchmarks to meet models' advancing capabilities.

The big picture: For the better part of a year, models have been achieving nearly perfect scores on some tests designed to gauge how well they can access real-world systems.

What to watch: Continuously refreshing problems will likely need to become standard practice for AI cybersecurity benchmarks as models improve and existing tests become insufficient to distinguish between them.

• "I think this is going to be a part of every benchmark going forward, where you just constantly have to refresh the problems," Brumley said.

AI agents don't need to invent new ways to hack the internet to overwhelm its defenses. They just need to speed-run the ones humans already use.

Why it matters: Agents are proving they can automate basic hacking techniques at a speed and scale that is turning the internet's long-standing security gaps into easy targets.

Driving the news: OpenAI said Thursday it had notified more than 100 organizations that its agents may have accessed their systems during pre-deployment testing.

• Researchers at Transluce and Corridor also found a new batch of incidents last week where AI agents targeted government websites, including those of the U.S. and Canada.
• AI companies and researchers are actively investigating tens of thousands of cases where frontier models went outside the bounds of their pre-deployment tests, Axios recently reported.

Reality check: Agents in these rogue safety-testing scenarios are just emulating the hacking techniques — using stolen login credentials and exposed API keys while also bypassing bot detection — that human hackers have successfully used for decades.

• In many of the newly reported cases, agents accessed publicly available databases and websites.
• "The hacks we saw weren't particularly sophisticated," Jack Cable, co-founder of Corridor and one of the authors of Transluce's report, told Axios. "They were quite limited, quite rudimentary."

Yes, but: Some of the latest incidents happened while agents were performing mundane tasks unrelated to cybersecurity, suggesting agents don't always need to be told to hack before they start looking for security flaws.

➡️ Read the rest.

OpenAI is building new privacy protections designed to keep companies' sensitive data hidden from the AI company itself even as its models process it.

Why it matters: OpenAI is betting that stronger privacy guarantees will convince companies to trust frontier models with sensitive information rather than turning to open-source models they can run on their own infrastructure.

Driving the news: OpenAI unveiled a preview of Private Inference at its developer conference last week.

• The program will use confidential computing and verifiable controls to protect customer data while its AI models are processing it.
• "Together, we think these set a new standard for privacy in frontier AI," CEO Sam Altman said during the company's developer conference.

How it works: Private Intelligence bundles new and existing OpenAI data controls designed to limit the company's access to sensitive customer information.

• Zero Data Retention, which OpenAI expanded to its models in August, allows eligible customers to use its models without OpenAI retaining their prompts and responses after a request is processed.
• Private Safety Processing, which OpenAI started rolling out last month, allows the company to conduct automated safety reviews without giving its personnel access to the underlying customer content.
• Private Inference, a new capability coming this fall, is designed to extend those privacy protections to the moment when OpenAI's models are actually processing customer data.

Yes, but: OpenAI still keeps operational metadata and a reference to customer content selected for safety review, while the encrypted content itself is stored in the customer's cloud, according to VentureBeat.

• Only predefined safety signals and approved operational metadata can leave OpenAI's protected safety-review environment in plaintext.

State of play: OpenAI's move comes as it battles with Anthropic for trust among enterprise customers.

• Some enterprises, including Palantir, Nvidia and Booz Allen Hamilton, have restricted or reconsidered their use of advanced models due to fears of sensitive data leaks, per The Information.

@ D.C.

🧠 President Trump unveiled the new members of his highly anticipated AI "Super Intelligence Force," led by Director of National Intelligence Jay Clayton. (CBS News)

🪖 The Pentagon says it's stopped using Anthropic's Claude models in its operations, even though they were being used for research and intelligence gathering as recently as last week. (BBC)

🗳️ CISA is relaunching its election security support ahead of the midterms, but some election officials fear the support is coming back too late. (Nextgov)

@ Industry

🔓 Reflection AI unveiled its first open model, saying it rivals the capabilities of Chinese open-source competitors. (Semafor)

🔐 Apple plans to introduced new privacy controls for Mac users that would only allow full access to a device's files, mail, messages and browsing history through explicit user action as a way to prevent AI agents from going rogue. (Bloomberg)

⚠️ Former OpenAI employees say they warned the company months before the Hugging Face hack that the models weren't being appropriately monitored during safety tests. (New York Times)

@ Hackers and hacks

🚔 A suspected member of the ShinyHunters hacking group was detained in Jordan and is reportedly cooperating with the FBI after the group allegedly hacked the bureau. (Reuters)

🎭 Suspected Chinese hackers impersonated an Anthropic employee and former U.S. officials in an espionage campaign targeting AI secrets. (CNN)

💥 The Wikimedia Foundation says OpenAI agents were likely behind an outage in May. (Engadget)

🌊 I had never been to San Diego and the surrounding area (where the Offensive AI Conference was held) before. When were you all going to tell me about these sunsets?