AI chip startup Groq grabs the spotlight
Add Axios as your preferred source to
see more of our stories on Google.
Illustration: Annelise Capossela/Axios
Groq, a Mountain View, Calif.-based artificial intelligence chip startup backed by the likes of Tiger Global and Lee Fixel's Addition, is gearing up to raise a fresh round next quarter after taking in a small recent cash infusion, the company tells Axios.
Why it matters: The company, which says its AI chips can be 10 times faster than conventional ones, is at the center of the recent firings at venture firm Social Capital.
The big picture: Founded in 2016 by Ross, Groq has built chips specifically designed for inference, that is, running generative AI models. It says its chips, dubbed "language processing units" (LPUs), are not only quicker but also one-tenth the cost of conventional AI hardware.
- The company currently has about 4,500 chips deployed, and Ross tells Axios it will have 1.5 million by end of next year.
- While at Google, Ross helped invent the company's tensor processing unit (TPU), its chip designed specifically for machine learning.
- To date, Groq has raised $367 million, most recently a $300 million round in 2021 at a $1.1 billion valuation led by Tiger Global and D1 Capital.
Zooming in: The startup's strategy is two-pronged. It's selling access to developers who want to run their AI software on its cloud infrastructure, and selling its hardware to customers operating their own data centers.
- It recently acquired Definitive Intelligence to underpin a new business division focused on expanding its offerings to customers and developers.
- The startup was founded in 2022 by Sunny Madra and Gavin Sherry, and backed by Social Capital, among others. (Madra and Sherry previously founded at least one other Social Capital portfolio company, Autonomic.)
- Groq's only other acquisition was Maxeler Technologies in 2022, though Ross says that more could definitely be in its future.
Between the lines: With the explosion in generative AI technology, applications and users, the demand for specialized computer chips has skyrocketed — but supply isn't quite keeping up.
- Groq and other upstarts are betting they can carve out a piece of the market, with Ross predicting that inference will overtake training when it comes to demand for compute resources — a perfect opportunity for his company.
- Groq is also pitching itself as a more environmentally friendly alternative to conventional infrastructure as concerns over AI's carbon footprint are also growing.
What they're saying: "It's clever marketing if you go out there and say that you're gonna spend $7 trillion, it makes people say 'Gosh, maybe we can't do this,'" Ross said of Sam Altman's public estimates of the capital OpenAI will need for a chip business.
- Ross says Groq could do the same for a mere $700 billion because its chips are so much cheaper.
Yes, but: In addition to capital, Groq still needs partners to execute its vision (and bear some of the upfront costs).
- It's already signed a deal with an unnamed partner to deploy over 10% of the 1.5 million LPUs it aims to get out by the end of 2025.
Inside the room: On the topic of Social Capital, Groq said in a statement that the events "are an internal matter at the firm about which we have very little knowledge at this point," adding that it will "determine how [Jay] Zaveri's departure from Social Capital will impact Groq's board of directors."
- It declined to clarify whether Zaveri, a former Social Capital partner, remains on its board.
What we're watching: Ross predicts that Nvidia's big annual conference next week will include significant talk about inference.
