AI hits trust hurdles with U.S. military

Driving the news: In a new essay in Foreign Affairs, Jacquelyn Schneider, the Hoover Institution's director of the wargaming and crisis simulation initiative, and AI researcher Max Lamparth write that large language models have a potentially dangerous tendency to go nuclear.

• When they tested LLMs from OpenAI, Anthropic and Meta in situations like simulated war games, the pair found the AIs suggested escalation, arms races, conflict — and even use of nuclear weapons — over alternatives.
• "It is practically impossible for an LLM to be taught solely on vetted high-quality data," Schneider and Lamparth write.

Zoom out: Older forms of machine learning-based AI are already deeply woven into the U.S. military, which uses it in everything from supply-chain analysis to interpreting satellite data.

• But the emergence of generative AI has happened on a lightning time-scale that has confounded the Pentagon.

Catch up quick: "The risk-taking appetite in Washington is not very great. And the risk-taking appetite out here [in Silicon Valley] is unparalleled," former Secretary of State Condoleezza Rice told Axios at a Hoover Institution media roundtable at Stanford University this week.

• The Space Force paused use of generative AI in September 2023.
• In June, the Navy's chief information officer, Jane Overslaugh Rathbun, concluded that while "generative AI can be a force multiplier," commercial models have "inherent security vulnerabilities" and are "not recommended for operational use cases."

Between the lines: Military use of AI faces challenges ranging from getting AI models to understand military jargon to concerns raised by lawyers over the handling of data, experts told Axios.

• Military and intelligence experts Axios spoke to said another big problem is that AI models can sift through vast quantities of data but they can't tell you how they arrived at any particular answer or suggestion. That makes it tough to base consequential decisions on their output.
• Generative AI is not good at dealing with unexpected events, like "Hamas attacking in a way no one thought Hamas could attack," Amy Zegart, senior fellow at Stanford's Hoover Institution, said.

Zoom in: The military's approach to data ownership makes it hard for anyone in the Pentagon to make the case for generative AI, even the new DoD chief digital and AI officer, the experts said.

• "The services own their own data, they own their own acquisition of technologies," Schneider said.
• "You have a team of lawyers that sit on those decisions. So we make it extremely complicated to be able to share data, acquire data, and to put all that data together" for generative AI implementation, per Schneider.

State of play: Booz Allen Hamilton on Tuesday released an open source version of its aiSSEMBLE platform, which aims to get government clients, including the DoD, out of AI "pilot purgatory" and into quicker deployment of AI.

• "There's a spectrum for use cases which DoD is evaluating through Task Force Lima," Ethan Wade, chief engineer and data scientist at Booz Allen, told Axios in an email. "From back office admin support for efficiencies (low stakes, high impact) to use cases directly affecting the warfighter (high stakes, high impact, where security and ethical constraints are much higher)."

Yes, but: Many AI critics argue that generative AI is too unreliable for social media use, let alone military applications. A slower Pentagon march toward using the new technology, in this view, is only prudent.

What they're saying: Rice said that the military only adopts leading edge technology when wartime demands force experimentation, or when "you get a few officers who are just like dogs on a bone on it."

• Schneider sees a contrast between "lots of experimenting" with AI at staff officer level but "no mechanism to take those lessons and then move it up or scale it," and says most senior officers are often "extremely naive" about the technologies.
• Alexandr Wang, Scale AI's CEO, told Axios that DoD and military branches need to take a step-by-step approach: "Testing and evaluating generative AI will help the DoD understand the strengths and limitations of the technology, so it can be deployed responsibly," he said.