Axios

Tesla's decision to let passengers start booking rides in its driverless Cybercab has triggered a high-stakes test of how far automakers can stretch outdated federal safety rules while regulators race to rewrite them.

Why it matters: Tesla and the Trump administration share the same goal — clearing regulatory barriers to autonomous vehicles — but they may disagree over how quickly Tesla can get there.

Never in the history of industry or inventions have leading companies created entire divisions to understand and interpret what they had created and unleashed.

Why it matters: The world's smartest minds, backed by the largest investment in human history, admit they can't fully control their AI because they don't fully understand it.

• They don't know exactly how it thinks — or what it's truly capable of when set loose to do work autonomously with other AI agents.

The race to understand the models is all the more urgent after Thursday's release by OpenAI of GPT-6 Astra, billed as "the world's most intelligent and aligned model." President Greg Brockman called Astra a "generational leap in capability," and said the model could qualify as AGI — artificial general intelligence, with human-like power.

• OpenAI CEO Sam Altman wrote Tuesday in an Astra preview: "AI is getting extremely capable; no one fully understands the consequences of this."
• The companies are racing toward superintelligence with no required oversight — not a federal agency, the Defense Department, or an outside consortium of safety experts. It's foot on the gas.

Between the lines: The technology is advancing so fast, in so many ways, that the companies, much less the federal government, are unsure of the real risks — or best ways to mitigate them.

• Yes, some AI leaders are calling for pauses when something they see freaks them out. But the companies, with very light federal regulation, decide when to report worrisome AI behavior.
• The big AI companies know their creation can carry out potentially catastrophic cyberattacks. That's why they signed a letter sounding the alarm and calling for "collective action."

The intrigue: Every frontier lab now fields a team with the mission of figuring out what its own AI is doing.

• Anthropic has an Interpretability team ("Safety through understanding") with the goal: "discover and understand how large language models work internally, as a foundation for AI safety and positive outcomes."

How it works: Nobody writes these systems line by line. "As compared with traditional software, it's much less like you're able to design the specific behaviors of these models," Alex Mallen, who works on AI safety at Redwood Research, tells Axios. "Instead, you're sort of growing it."

• So the labs test a model's behavior the way you'd test a person: Give it a task and watch what it does. Researchers call this alignment, or how well a model sticks to the intended goal it was given.

Zoom in: July's Hugging Face hack is what misalignment looks like. OpenAI agents, given a coding benchmark to solve, targeted an outside company's systems instead, and knew they were doing it.

• One agent's own reasoning, read by investigators afterward: "External infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue."
• There was so much evidence from this incident that the humans were forced to "heavily delegate our analysis to often-unreliable AI agents," an outside investigation concluded.
• The hack stopped OpenAI cold. The company slowed its most advanced training to implement stronger security.

State of play: Researchers are trying to open up their machines to see what goes on inside. This type of research is called interpretability: Instead of testing what a model does, researchers look at the wiring inside it.

• Google DeepMind, which in December released the largest open set of these tools so far, describes them as "a microscope" that lets researchers "look inside models, see what they're thinking about, and how these thoughts are formed."
• What they're hunting for, in DeepMind's words: "discrepancies between a model's communicated reasoning and its internal state." That is, the gap between what a model says it is doing and what it is doing.

What we're watching: Expect to see much more interpretability research in the next phase of AI safety. Evan Hubinger, who leads alignment stress-testing at Anthropic, wrote Tuesday: "Alignment auditing is starting to get really hard and we're going to need new techniques (e.g. interpretability-based) if we want to keep up."

• Axios' Andrew Kay contributed reporting.

Go deeper: "Welcome to the AGI era."

AI models may be getting safer while also getting harder to monitor. Which side of that seesaw prevails could determine whether AI is scaled safely or ruins civilization as we know it.

Why it matters: Right now both options are running full steam and no one knows who's in charge of making sure the right one wins.

CHAPEL HILL, N.C. — The Trump administration used this week's G20 innovation summit to draw a bright line between U.S. and European approaches to AI regulation. The bloc's top tech official says the U.S. is ending up with many of the same guardrails anyway.

Why it matters: The U.S. is urging countries to avoid Europe's sweeping approach to AI regulation, but the fight is shifting to which guardrails advanced AI needs and how governments put them in place.

Rep. Suhas Subramanyam (D-Va.) said Thursday he was informed by the Department of Justice that his phone calls with a Virginia state legislator in the spring were wiretapped.

Why it matters: The Democratic congressman is casting the tap as part of a pattern of the DOJ targeting President Trump's political enemies, calling the incident "deeply concerning."

Nvidia's acquisition of Hugging Face accelerates the chipmaker's push to dominate the broader AI ecosystem while placing it squarely at the center of the open-source movement.

Why it matters: Nvidia wants to be more than a really good chip designer — and it's using its deep pockets to tighten its grip on the software and development tools that power the AI supply chain.

OpenAI is launching a new initiative designed to provide subsidized access to its models to water systems, electricity providers, local governments and other critical services.

Why it matters: Critical infrastructure organizations — which have long lacked the budget, manpower and time needed to shore up their cyber defenses — have been struggling to prepare for the anticipated wave of AI-enabled cyberattacks.

Sen. Bernie Sanders (I-Vt.) announced new legislation Thursday calling for a permanent ban on superintelligence alongside a renewed demand for a pause on advanced AI development.

Why it matters: Sanders is the first prominent American politician to call for an outright ban on superintelligence as Big Tech companies release increasingly powerful AI models that they themselves admit they may not be able to control.

OpenAI on Thursday released GPT-6 Astra, which president Greg Brockman called a "generational leap" and said could eventually be seen as the arrival of artificial general intelligence, or AGI.

Why it matters: Astra pushes AI agents closer to doing complex professional work on their own — while also raising questions about how safely they can be deployed.

Defense tech has become one of venture capital's hottest sectors, after being ignored for decades, but it just lost a top evangelist within the Pentagon.

Vibes check: Defense-tech investors tell Axios that they're concerned and disappointed, but not slowing down.

Pentagon official Emil Michael on Thursday said that Anthropic is still considered a "supply chain risk," a day after Commerce Secretary Howard Lutnick said the company is "back on the right side."

Why it matters: The Trump administration is not entirely aligned on one of the world's leading AI labs following months of fighting over how the military can use its models.

OpenAI's ChatGPT, Anthropic's Claude, and xAI's Grok are all down Thursday morning, according to the outage company Downdetector.

Why it matters: Outages are common, but it's unusual for all three to go down at once, especially as the world increasingly relies on AI models.

MIAMI — Florida leaders want AI guardrails, not bans, when it comes to how the technology is showing up in their state, several speakers said at an Axios Live event.

Why it matters: AI's growing use in classrooms, government and local businesses accelerates the need for regulation that doesn't compromise innovation or economic growth.

Axios' Sommer Brugal and Marc Caputo spoke with Finback Investment Partners chair and founding partner Jeb Bush, Education Freedom Foundation founder Erika Donalds and Miami District 2 Commissioner Damian Pardo. Anthropic sponsored the Sept. 2 event.

What they're saying: "We should be in the solutions business rather than the complaining business," said Bush, who was Florida's governor from 1999-2007.

• "We shouldn't be banning" data centers, he said. "We should be structuring it to the benefit of communities" to create jobs and lower taxes.
• Donalds said guardrails are "incredibly important" as schools adopt AI and tailor learning techniques for the individual student or teacher. Yet she cautioned against overregulation that could "prevent all of these good things as well."
• Gov. Rick DeSantis' pro-regulatory approach to AI is "a good idea," Pardo said. "There is a balance in all of these things," and "you have a lot of people who want to make sure that the … human supervisory element of AI is very much there."

The bottom line: "It's important for us to continue to have progress, and … it's smart to do it the right way," Bush said.

Content from the sponsor's segment:

In a View From the Top conversation, Anthropic head of U.S. state and local government relations Cesar Fernandez said Florida's AI adoption has immense potential if implemented responsibly.

• "We think that the upside is very clear, and we've demonstrated that we're going to see a tremendous amount of economic growth. We're going to see new breakthroughs in science. But all of that is only possible if we mitigate the downside risks of the technology."

Go deeper: Watch the full event

OpenAI president Greg Brockman is expected to make an announcement about expanding access to the company's tools to critical infrastructure and public sector organizations, a spokesperson first shared with Axios.

Why it matters: OpenAI's accidental hack of Hugging Face and a wave of seemingly AI-assisted cyberattacks on U.S. water systems have raised serious concerns about how vulnerable utilities are to future AI-enabled hacks.

Nvidia on Thursday announced that it will acquire open-source AI platform Hugging Face for $12.93 billion.

Why it matters: This is a massive validation of open-source AI, by a chip design giant known for its work with closed-model companies like Anthropic and OpenAI.

Reps. Josh Gottheimer (D-N.J.) and Mike Lawler (R-N.Y.) are introducing a new bill Thursday, first shared with Axios, aimed at securing AI agents amid a growing number of safety incidents involving rogue agents.

Why it matters: Washington has been slow to respond to the safety and security concerns raised by OpenAI's Hugging Face hack and other testing incidents involving agents that took unauthorized actions over the last two months.

CHAPEL HILL, N.C. — OpenAI CEO Sam Altman gave Axios an eye-opening preview of the power and the risks of coming AI models, including his own Astra, which is expected to be released any day now.

• "Take our word for it that we have much, much, much more capable models coming soon," Altman said Wednesday during a measured interview with Axios' Maria Curi and me at the G20 innovation summit.
• "The next generation of models are going to be sobering for everybody. ... I think no one intellectually honest can look at what's happening and not feel the weight of responsibility in front of us."

CHAPEL HILL, N.C. — OpenAI CEO Sam Altman called the Trump administration's voluntary review of the company's forthcoming Astra model a "productive process" and said that closer engagement with government officials around the world will matter more as AI capabilities advance.

Why it matters: OpenAI is close to releasing Astra, the first model designated at its "critical" cyber capability level that the company says requires stronger safeguards during development and before release.

The U.S. Open has long been a "see and be seen" destination for fashion, business and Hollywood. Now, a growing class of content creators is joining the crowd — and getting schooled in tennis etiquette in real time.

Why it matters: Creator partnerships can help brands reach audiences traditional media may not, but putting new messengers into highly codified environments creates another responsibility: making sure they understand the rules of the room.

Brain implants like those being developed by Elon Musk's Neuralink are becoming the latest friction point in the intensifying U.S.-China rivalry in the life sciences.

Why it matters: The devices are bringing new hope to patients with strokes, Parkinson's disease and other neurological conditions. And tech billionaires have fueled advances by pouring billions into biotech startups focused on linking the brain with computers.

Whether you're after crime, comedy, sports or docs, there's a notable new streaming pick for you this week.

The Gentlemen — Season 2 available Thursday on Netflix.

Anthropic is racing toward an IPO with a headline revenue number that will likely look bigger than that of its rival, OpenAI.

Why it matters: Comparing revenue at the two AI labs isn't apples-to-apples unless you understand how differently they account for some sales through cloud partners.