California's AI safety squeeze
Add Axios as your preferred source to
see more of our stories on Google.
Illustration: Aïda Amer/Axios
As states take the AI legislation lead from a deadlocked Congress, a new California AI safety bill is pitting some leaders of the state's hottest industry against many of its lawmakers.
The big picture: Proponents of the bill say AI is advancing at such a rapid pace that California shouldn't wait for Washington to establish protections for the public. Opponents maintain that the rules will stifle AI innovation.
What they're saying: "In the tech sector, there are people who just don't want any regulation," California state Sen. Scott Wiener said Tuesday when asked about the pushback against the bill he authored, known as the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act.
Catch up quick: The bill requires makers of advanced or "frontier" AI models to conduct safety testing and certify their products' safety.
- It would empower the state's attorney general to take legal action against AI makers if their technology causes "critical harms," and especially any "mass casualty event."
- It only applies to AI models that surpass a very high threshold for how much computing power and money was used to train and operate them.
- It includes a "kill switch" provision, requiring AI makers to be able to shut down an advanced AI at any time.
- It also creates a Frontier Model Division within the state's Government Operations Agency to determine and enforce the new safety standards.
What they're saying: "This bill, as it stands, could gravely harm California's ability to retain its AI talent and remain the location of choice for AI companies," around 140 AI startup founders said in a letter authored by Y Combinator.
- Other critics are concerned that it would dampen the development of open source AI models.
The other side: "We're setting the threshold for models that frankly don't exist and won't soon," said Wiener, who spoke at the AIQCon in San Francisco via (malfunctioning) video chat.
- He added that there's been a lot of misinformation about the bill, namely that it would apply to AI startups, and that it would result in software "developers going to jail."
- Wiener said that he's also considering the concerns of the open source tech community, which he praised as "incredibly important."
Between the lines: Supporters and critics of the bill disagree on one or more of these thorny questions: Can AI become dangerous to humanity? Should we be regulating a technology or only its uses? And does regulation inherently stifle innovation?
- Some top AI names who have long warned of the technology's existential risks have come out in support of the bill, including AI pioneers Geoffrey Hinton and Yoshua Bengio.
The intrigue: As Wiener pointed out, California is making moves in this area because Washington hasn't, though he's hoping it will.
- "I think in a lot of areas it's ideal to have federal standards, but I'm not confident that Congress will act on AI regulation," he said.
- "I hope I'm wrong, but the reality is that in the year 2024, there's no federal data privacy law. In 2024, other than banning TikTok, Congress has done nothing on social media."
What's next: After making it out of the state assembly's Privacy Committee, the bill is headed for consideration by the Judiciary Committee next week.
- Wiener's is only one of a raft of different bills under consideration in California governing different aspects of AI, including privacy, data transparency, copyright and bias.
