AI is repeating A-bomb's story, says EU tech overseer

• "If we deal with individual existential threats first, we'll have a much better go at dealing with existential threats towards humanity," Vestager told Axios.
• Humans have never before been "confronted with a technology with so much power and no defined purpose," she said.
• The Institute for Advanced Study was famously led by J. Robert Oppenheimer from 1947 to 1966, after his Manhattan Project developed the world's first nuclear weapons.

The big picture: In her lecture, Vestager — whose full title at the European Commission is "executive vice president for a Europe fit for the digital age and competition" — forcefully argued that "technology must serve humans."

Friction point: Vestager's era at the EU has coincided with passage of some of the world's most comprehensive tech regulations and the pursuit of a raft of enforcement actions against tech giants. But she rejects the belief, held by many in the industry, that this approach has hobbled European innovators and economies.

• "We regulate technology because we want people and business to embrace it," she said, including Europe's "huge public sectors."
• Europe's biggest tech problem is companies not scaling, she said, blaming "an incomplete capital market" and, in the case of AI startups, trouble accessing necessary chips and computing power.

She also maintains that she has not bullied companies into applying EU regulations globally, as some have suggested.

• "We're not trying to de facto legislate for the entire world. That would not be proper," she said. But she urged U.S. legislators and AI founders and engineers to "interact with the outside world" to uphold their responsibility to humanity.

Trust is a big problem for AI companies, according to Vestager — echoed by a long list of opinion polls and surveys.

• "Trust is something that you build when you also have something to keep you on track," she said, "like the EU AI office and the U.S and U.K. Safety Institutes."
• "Governments can set benchmarks," but "it's really important that a red-teaming sector develops," she added.
• Vestager said that rights to fairness and transparency around decisions made with AI are meaningless if they cannot be enforced through rules.

In her speech, Vestager said that large digital platforms are "challenging democracy," but that "general purpose artificial intelligence" is "challenging humanity."

• "With AI, you can even give up on relationships" with people, she said, citing the rise of robot and chatbot companions. "And if we lose relationships, we lose society. So we should never give up on the physical world."
• "I compare this moment to 1955," she told Axios, referring to the time when the cost of inaction on nuclear safety had become too high for any country to ignore, and forced nuclear powers to come together to protect humanity.

• The International Atomic Energy Agency was created in 1957, "which then created the conditions for the Nuclear Non-Proliferation Treaty," she said.

What's next: Vestager wants universal governance on AI safety, even though that means compromising with governments "we fundamentally disagree with."

• She sees the U.S.-EU Trade and Technology Council as a place to generate ideas for other forums, saying the G7 code of conduct on AI grew out of TTC discussions.
• The G7 code now "needs to be swiftly put in motion through a common monitoring and evaluation system."

The U.S. can't beat China alone, in Vestager's view.

• Democratic governments need to fund strategic investments in fields like AI and clean tech, while managing their markets to include any company that meets certain "trustworthiness principles," she said.
• That's a way to protect political and economic interests "as the EU and U.S. have already done for 5G vendors," without generating expensive subsidy wars or telling Chinese companies they are never welcome.
• Today we say "you cannot compete while destroying the environment, or destroying workers. So we [already] organize the marketplace around things we do not accept," Vestager said. We have to apply those values to Chinese companies in our markets, she maintained, so they "cannot destroy our economic security."

Meanwhile: Vestager lashed out at Apple for what she sees as a deliberate attempt to skirt new EU rules for "gatekeeper" tech companies.

• "I think they know exactly what they do," Vestager said, claiming a "strong suspicion" that Apple's new App Store rules in the EU don't comply with the Digital Markets Act.