As states take the AI legislation lead from a deadlocked Congress, a new California AI safety bill is pitting some leaders of the state's hottest industry against many of its lawmakers.

The big picture: Proponents of the bill say AI is advancing at such a rapid pace that California shouldn't wait for Washington to establish protections for the public. Opponents maintain that the rules will stifle AI innovation.

What they're saying: "In the tech sector, there are people who just don't want any regulation," California state Sen. Scott Wiener said yesterday when asked about the pushback against the bill he authored, known as the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act.

Catch up quick: The bill requires makers of advanced or "frontier" AI models to conduct safety testing and certify their products' safety.

• It would empower the state's attorney general to take legal action against AI makers if their technology causes "critical harms," and especially any "mass casualty event."
• It only applies to AI models that surpass a very high threshold for how much computing power and money was used to train and operate them.
• It includes a "kill switch" provision, requiring AI makers to be able to shut down an advanced AI at any time.
• It also creates a Frontier Model Division within the state's Government Operations Agency to determine and enforce the new safety standards.

What they're saying: "This bill, as it stands, could gravely harm California's ability to retain its AI talent and remain the location of choice for AI companies," around 140 AI startup founders said in a letter authored by Y Combinator.

• Other critics are concerned that it would dampen the development of open source AI models.

The other side: "We're setting the threshold for models that frankly don't exist and won't soon," said Wiener, who spoke at the AIQCon in San Francisco via (malfunctioning) video chat.

• He added that there's been a lot of misinformation about the bill, namely that it would apply to AI startups and would result in software "developers going to jail."
• Wiener said that he's also considering the concerns of the open source tech community, which he praised as "incredibly important."

Between the lines: Supporters and critics of the bill disagree on one or more of these thorny questions: Can AI become dangerous to humanity? Should we be regulating a technology or only its uses? And does regulation inherently stifle innovation?

• Some top AI names who have long warned of the technology's existential risks have come out in support of the bill, including AI pioneers Geoffrey Hinton and Yoshua Bengio.

The intrigue: As Wiener pointed out, California is making moves in this area because Washington hasn't, though he's hoping it will.

• "I think in a lot of areas it's ideal to have federal standards, but I'm not confident that Congress will act on AI regulation," he said.
• "I hope I'm wrong, but the reality is that in the year 2024, there's no federal data privacy law. In 2024, other than banning TikTok, Congress has done nothing on social media."

What's next: After making it out of the state assembly's Privacy Committee, the bill is headed for consideration by the Judiciary Committee next week.

• Wiener's is only one of a raft of different bills under consideration in California governing different aspects of AI, including privacy, data transparency, copyright and bias.

The U.S. Digital Service is looking to AI as it helps the federal government improve its tech operations.

Why it matters: The agency is in charge of consulting the government on IT and how to improve the websites and apps that many Americans rely on, such as the IRS' tax filing systems and the Department of Veterans Affairs' services portal.

Driving the news: We sat down with USDS administrator Mina Hsiang at the Collision conference in Toronto last week, where she spoke about how to use tech to strengthen democracy.

• Hsiang, an engineer, helped rescue the HealthCare.gov website in 2013 when Americans across the country were struggling to get health insurance after the passage of the Affordable Care Act.

Now, with the explosion of AI, Hsiang said she sees many opportunities for fraud detection, predictive analytics and tools to give customer service "superpowers."

• For example, USDS is exploring opportunities for AI to assist customer service representatives to answer questions faster through tools that aid in onboarding and training, Hsiang said.
• USDS has been using AI to analyze large volumes of call center data to help classify calls and identify drivers of wait times at the Social Security Administration.
• And USDS is working with the Department of Homeland Security on AI framework pilots, including helping FEMA with disaster preparedness planning.

Our thought bubble: Obtaining government services online can be slow and frustrating, but AI could improve that experience.

• Hsiang said humans will still be involved in making important decisions and that USDS is looking to hire experienced technologists.

A version of this story was published first on Axios Pro. Unlock more news like this by talking to our sales team.

• OpenAI pushed back the initial launch of its latest voice assistant by a month over safety issues. Meanwhile, it made the ChatGPT app for MacOS available to all users. (Bloomberg, 9to5Mac)
• Cruise named former Microsoft and Amazon executive Marc Whitten as its new CEO. (CNBC)
• Character.ai users are reporting that their chatbot characters are showing less personality than before. (404 Media)

I just learned about this method of folding shirts using a cardboard box and am legit obsessed.