A surprising new study finding that AI tools can reduce programmers' productivity is upending assumptions about the technology's world-changing potential.

Why it matters: Software runs our civilization, and AI is already transforming the business of making it — but no one really knows whether AI will decimate programming jobs, or turn every coder into a miracle worker, or both.

Driving the news: The study by METR, a nonprofit independent research outfit, looked at experienced programmers working on large, established open-source projects.

• It found that these developers believed that using AI tools helped them perform 20% faster — but they actually worked 19% slower.
• The study appears rigorous and well-designed, but it's small (only 16 programmers participated, completing 246 tasks).

Zoom out: For decades, industry visionaries have dreamed of a holy grail called "natural language programming" that would allow people to instruct computers using everyday speech, without needing to write code.

• As large language models' coding prowess became evident, it appeared this milestone had been achieved.
• "The hottest new programming language is English," declared AI guru (and OpenAI cofounder) Andrej Karpathy on X early in 2023, soon after ChatGPT's launch.
• In February, Karpathy also coined the term "vibe coding" — meaning the quick creation of rough-code prototypes for new projects by just telling your favorite AI to whip up something from scratch.
• The most fervent believers in software's AI-written future say that human beings will do less and less programming, and engineers will turn into some combination of project manager, specifications-refiner and quality-checker.
• Either that, or they'll be unemployed.

Zoom in: AI-driven coding tends to be more valuable in building new systems from the ground up than in extending or refining existing systems, particularly when they're big.

• While innovative new products get the biggest buzz and make the largest fortunes, the bulk of software work in most industries consists of more mundane maintenance labor.
• Anything that makes such work more efficient could save enormous amounts of time and money.

Yes, but: This is where the METR study found AI actually slowed experienced programmers down.

• One key factor was that human developers found AI-generated code unreliable and ended up devoting extra time to reviewing, testing and fixing it.
• "One developer notes that he 'wasted at least an hour first trying to [solve a specific issue] with AI' before eventually reverting all code changes and just implementing it without AI assistance," the study says.

Between the lines: The study authors note that AI coding tools are improving at a rapid enough rate that their findings could soon be obsolete.

• They also warn against generalizing too broadly from their findings and note the many counter-examples of organizations and projects that have made productivity gains with coding tools.

One notable caution that's inescapable from the study's findings: Don't trust self-reporting of productivity outcomes. We're not always the best judges of our own efficiency.

• Another is that it's relatively easy to measure productivity in terms of "task completion" but very hard to assess total added value in software-making.
• Thousands of completed tickets can be meaningless — if, for instance, a program is about to be discontinued. Meanwhile, one big new insight can change everything in ways no productivity metric can capture.

The big picture: The software community is divided over whether to view the advent of AI coding with excitement or dread.

• "Quitting programming as a career right now because of LLMs would be like quitting carpentry as a career thanks to the invention of the table saw," the AI developer and blogger Simon Willison recently wrote on Bluesky.
• Or "it could be like quitting agriculture as a career thanks to the invention of tractors and combine harvesters instead," industry veteran Tom Coates replied.

Elon Musk's xAI has secured a contract worth up to $200 million with the U.S. Department of Defense, the company announced yesterday.

The big picture: The move is a part of the Pentagon's adoption of advanced artificial intelligence capabilities to address national security challenges.

• As part of that goal, the Chief Digital and Artificial Intelligence Office, which sets AI standards for the Defense Department, also announced awards to Anthropic, Google and OpenAI.

Zoom in: xAI also launched a suite of products making its flagship model, Grok, available to the U.S. government.

• "Under the umbrella of Grok For Government, we will be bringing all of our world-class AI tools to federal, local, state, and national security customers," the company said.
• "These customers will be able to use the Grok family of products to accelerate America — from making everyday government services faster and more efficient to using AI to address unsolved problems in fundamental science and technology."

What they're saying: "This allows every federal government department, agency, or office, to access xAI's frontier AI products," the company said.

• Douglas Matty, chief digital and AI officer at the Defense Department, said in a statement that the adoption of AI is "transforming the Department's ability to support our warfighters and maintain strategic advantage over our adversaries."

Cognition, the startup behind coding assistant Devin AI, has agreed to acquire Windsurf, it announced yesterday.

Why it matters: The parts of Windsurf that weren't snapped up by Google — including employees — have a new home.

Context: OpenAI was previously in talks to acquire Windsurf, but those talks collapsed. Google swooped in and revealed Friday that it had agreed to acqui-hire some individuals, including CEO Varun Mohan, in a $2.4 billion licensing deal.

• According to a note to employees from Cognition CEO Scott Wu, all Windsurf employees will participate financially in their deal and receive fully accelerated vesting for their work to date.
• Terms of the deal weren't revealed.

Zoom in: OpenAI's almost-deal for Windsurf widened cracks in its relationship with Microsoft, with OpenAI and Microsoft dueling over whether the latter would have access to Windsurf's IP.

The big picture: Regulatory scrutiny around Big Tech — and incestuous alliances between the AI companies — are making such quasi-mergers a staple of the industry.

Context: This deal echoes Microsoft's $650 million deal for Inflection AI last year, where Inflection exists in name, but its brains — CEO Mustafa Suleyman and key employees — would join Microsoft.

• Google inked a similarly structured deal with Character.ai last year, and Meta made a $15 billion deal for a 49% stake in Scale AI earlier this year that resulted in CEO Alexandr Wang joining the social media company.

• Nvidia can start selling AI chips to China again, after facing export controls. (Axios)
• The EU has laid out the rules for AI firms like OpenAI and Google navigating the new AI Act, with respect to transparency, copyright, and safety requirements. (Axios Pro)
• The superintelligence team at Meta is reportedly weighing whether the company should replace its open source model with a more opaque, closed model. (New York Times)
• Nextdoor is rebooting its app and adding AI-generated summaries of neighborhood recommendations and AI-generated prompts to spark discussion. (Axios)

The word would probably just be an em dash.