AI's shaky foundations
Add Axios as your preferred source to
see more of our stories on Google.
/2021/08/18/1629305368046.gif?w=3840)
Illustration: Sarah Grillo/Axios
A new academic group is sounding a warning about powerful, if poorly understood, AI systems that are increasingly driving the field.
Why it matters: New models like OpenAI's text-generating GPT-3 have proven so impressive that they're serving as the foundation of further AI research, but that risks propagating the biases that may be built into these systems.
What's happening: This morning a group of more than 100 researchers released a new report on the "opportunities and risks" of foundational AI systems as part of the launch of a new group at Stanford's Institute for Human-Centered AI called the Center for Research on Foundation Models (CRFM).
- The report warns that the very qualities that have made these models so exciting — and potentially so commercially valuable — creates what Percy Liang, a Stanford computer science professor and the director of CRFM, calls "a double-edged sword."
- "We're building AI infrastructure on a handful of models," he adds, but our inability to fully understand how they work or what they might do "means that these models actually form a shaky foundation."
Background: Liang notes that until recently, AI systems were built for specific purposes — if you needed machine translation, you built a machine translation model.
- But that began to change in 2019, when Google introduced its BERT natural language processing (NLP) model.
- BERT now plays a role in most of Google's search functions, while Facebook researchers harnessed BERT as the basis for an even larger NLP model that it uses for AI content moderation.
- At the same time companies like OpenAI and AI21 Labs have begun allowing developers to build commercial applications off their own massive NLP systems.
How it works: With these systems, "you just grab a ton of data, you build a huge model, and then you go in and discover what it can do," says Liang.
- As an AI scientist, he adds, the power of these models is "so cool," but they also risk homogenizing the AI field.
- Any biases in these models — or in the data they're built upon — "risks being amplified in a way that everyone inherits," says Liang.
The bottom line: The good news is this foundation is still being built, so interdisciplinary groups like CRFM can work to study those defects and hopefully correct them.
- The bad news is that we may be running out of time to do just that.
Editor's note: This story was updated to clarify that the Center for Research on Foundation Models (CRFM) is at Stanford's Institute for Human-Centered AI.
