OpenAI fires up "Jalapeño," its first homegrown AI chip
Add Axios as your preferred source to
see more of our stories on Google.
Broadcom CEO Hock Tan delivers a chip wafer to OpenAI CEO Sam Altman. Image: OpenAI
OpenAI said Wednesday that it has begun testing "Jalapeño," the first in a family of homegrown chips, with plans to start using the chips to handle customer queries later this year.
Why it matters: OpenAI is joining other leading AI companies in designing its own silicon as it races to secure more computing capacity, lower costs and reduce its dependence on Nvidia.
Driving the news: This first generation of chips, which are aimed at inference rather than training, were developed with some help from Broadcom.
- OpenAI did the core design, while Broadcom brought specific knowledge in connectivity and other areas.
- OpenAI said its chips are specifically designed for handling current and future models, allowing them to be used more efficiently and deliver better performance per watt of electricity than off-the-shelf options.
Zoom in: OpenAI is using the first sample chips in its labs for tasks similar to answering Codex queries.
- They're delivering even better thermal performance than anticipated, the company says.
- Broadcom said to expect the first chips to be in commercial use at Microsoft and other partners by the end of the year, though OpenAI says the real volume will come next year.
- The company has said it aims to have the custom chips powering 10 gigawatts' worth of compute by 2029.
What they're saying: Broadcom CEO Hock Tan said that the work with OpenAI highlights a message he has been pushing for a while: Companies that want to lead in AI need their own chips.
- "At the end of the day, you cannot, should not rely on some other third-party GPU to do it for you, because it's such a key part," Tan said.
- Richard Ho, who leads OpenAI's chip efforts, said that being able to bring AGI to benefit all of humanity requires the company to deliver compute efficiently and cost-effectively.
- "This gives OpenAI full stack control," Ho said.
The big picture: OpenAI has relied almost exclusively on Nvidia chips for both training and inference, though recently it began also using chips from Cerebras for inference.
- Nvidia remains a key partner, especially for training new models, OpenAI says.
- Google, Amazon, Microsoft and others already have their own AI chips that they use alongside Nvidia's processors.
What we're watching: Whether OpenAI expands its homegrown chip use to training — not just inference, something the company said it is considering.
- "We think we have a very efficient architecture and so we will try to maximize our architecture in different dimensions," Ho said.
