Exclusive: Gary Marcus says LLMs are a "dress rehearsal" for AGI
Add Axios as your preferred source to
see more of our stories on Google.
Gary Marcus at Axios Live. Photo: Chris Constantine on behalf of Axios
Gary Marcus said large language models (LLMs) are useful, but their real talent is building the foundation for a future powered by artificial general intelligence during an appearance Thursday at Axios' AI+ Summit.
Why it matters: Three years after the release of OpenAI's ChatGPT, Marcus argues that LLMs aren't delivering the revolution developers promised, and they probably never will — but artificial general intelligence (AGI) could.
What they're saying: "The...problem with LLMs is you can't really align them," he said to Axios' Ina Fried, noting that they lack the knowledge of world models.
- "You put in stuff like, 'don't hallucinate,' and they hallucinate anyway. If you say, 'don't do harm to humans,' they do anyway, or at least potentially do anyway. So we need a much stronger solution to the alignment problem," he said.
- "I think LLMs are like a dress rehearsal, right?"
- "They're not AGI. Anybody who thinks they are is just not really following the technical detail well enough, I think. But they give us a chance to think about what would the world be like if we had AGI."
Context: Marcus isn't shy about his belief that LLMs have significant pitfalls.
- He's concerned about their tendency towards bias, tendency to spit out misinformation or hallucinate, copyright considerations, and impact on the environment and education.
- He also worries about AI contributing to the "gradual enshittification of the internet" — the decay of sites and platforms as they are overwhelmed with slop.
- He thinks these problems are baked into the technology and aren't going away.
State of play: The AI industry has long moved with a "move fast and break things" mentality, and there's really no sign that companies or the government are concerned with significantly regulating AI's allegedly unlimited capabilities.
- AI executives frequently admit they have no idea why their products do what they do, but they're going to keep investing more money into the dream of achieving artificial general intelligence one day — a dream that is always pushed further into the future each time you ask for a timeline.
- Nonetheless, the industry and the Trump administration are bullish on developing AI as fast as possible to keep American technology ahead of China's.
Go deeper: AI "bubble" charge misses the capacity for change, investors say
