Allen Institute for AI releases fully open-source large language model
Add Axios as your preferred source to
see more of our stories on Google.
Image: Allen Institute for AI
The Allen Institute for AI (AI2) released a fully open-source large language model designed to help researchers better understand what's taking place under the hood.
Why it matters: The move comes as some have argued open-source alternatives are needed to avoid concentration of power, while others worry that such models could be harder to regulate.
Details: OLMo 7B, as the model is known, was created with support from AMD, Databricks, and researchers at Harvard and the University of Washington.
- AI2 (created by Microsoft co-founder Paul Allen) is going further than most companies offering open source models, releasing not only the model and its weights, but also its full training data and pre-training code.
- OLMo 7B will be available for direct download on Hugging Face and via GitHub.
- OLMo stands for "open language model."
Between the lines: Releasing so much data, along with the models themselves, allows researchers to better understand how such systems work, the Allen Institute says.
- For example, one can see whether a model has truly learned a new set of skills or just memorized answers to a particular test.
- More open models, which include training and evaluation systems, could also eliminate redundancies that drive up environmental impacts of AI, the Allen Institute says, noting that a typical training run for a large language model is equivalent to full-year emissions from nine U.S. homes.
What they're saying: AI2 CEO Ali Farhadi says having a truly open, state-of-the-art large language model will "fundamentally change how researchers and developers learn about and build AI."
- "Access to truly open models has never been more critical for the development of AI," Farhadi said in a statement to Axios.
- While commercialization isn't inherently bad, Farhadi said the limited transparency offered by many large language models made it hard to assess performance as well as issues related to toxicity and bias.
- Executives from Meta and AMD praised the Allen Institute's effort. "Open foundation models have been critical in driving a burst of innovation and development around generative AI," Meta chief AI scientist Yann LeCun said in a statement.
- Meta has been a vocal advocate for open-source AI and has released its Llama 2 model for commercial use. "The vibrant community that comes from open source is the fastest and most effective way to build the future of AI," LeCun said.
