New "compute grid" takes aim at chip supply crunch
Add Axios as your preferred source to
see more of our stories on Google.
Illustration: Lindsey Bailey/Axios
A coalition of AI startups, cloud providers, researchers and investors is launching a National Compute Grid, an effort to pool AI computing to help address a supply crunch driving enormous spending on AI infrastructure.
Why it matters: Building AI infrastructure has become a multi-trillion-dollar bet largely because of the high cost of accessing scarce computing resources. Yet many data centers run at low capacity, meaning extremely valuable chips sit idle while some startups and researchers struggle to get access.
- The creators of the compute grid want to make it easier for idle computing capacity to be used, a goal that could significantly alter the direction of the AI boom if it were to take off.
What they're saying: Anjney Midha, a leader of the effort, told Axios the project is a way to share computing for both commercial research and public-sector use. "The best way to scale AI in America efficiently, and stay at the frontier and stay competitive with China, is to be" coordinated around an open standard, he said.
- "Turns out, we actually do have a lot more compute than people expect. It just all needs to be interconnected. And coordinated," said Midha, a former venture investor who now runs AI holding-company Amp, a public benefit corporation.
- Sam Sinha, Head of AI at 1X, a startup focused on embodied AI for humanoid robots, said AI companies like his often struggle to get access to computing resources. Larger players like OpenAI and Anthropic can pay far more, and sign up for long-term contracts that aren't possible for smaller operators, he said.
- "We need to encourage a healthy AI ecosystem, and have more than two companies to own all the compute," he said, referring to OpenAI and Anthropic.
Between the lines: AI companies need vast computing resources while training new AI systems and often come close to fully using their capacity during training runs. Outside of those runs, many chips sit idle.
- The consortium says independent, single-tenant data centers average less than 15% net computing utilization, leaving expensive capacity unused even as researchers face shortages, according to a paper it produced with the announcement.
How it works: The National Compute Grid plans to pool AI computing capacity from labs, cloud providers and different chip systems, then allocate it through a shared scheduler.
- The system is designed to show members available capacity, chip type, location, pricing and utilization, then automate how workloads are matched to it.
- Grid members can contribute idle capacity and reserve larger clusters for planned training runs.
- The consortium says it has about 760 megawatts connected or in sight, with a goal of 2 gigawatts by 2030.
What's next: The paper says the grid is opening access to public-sector employees and teams, including government, education and national laboratory users.
The bottom line: The primary goal to make computing more broadly available is just a start. Finding ways to bring more existing capacity online also has the potential to increase supply of one of the scarcest, and most precious, resources in the world.
