Axios AI+

March 27, 2024
Ina here, catching my breath after an amazing weekend of NCAA women's hoops that busted my bracket but did my heart good. Today's AI+ is 1,184 words, a 4.5-minute read.
1 big thing: Chatbot letdown as hype hits rocky reality
Illustration: Brendan Lynch/Axios
Grumbles about generative AI's shortcomings are coalescing into a "trough of disillusionment" after a year and a half of hype about ChatGPT and other bots.
Why it matters: AI is still changing the world, but improving and integrating the technology is raising harder and more complex questions than first envisioned, and no chatbot has the magic answers.
Driving the news: The hurdles are everything from embarrassing errors — such as extra fingers or Black founding fathers in generated images — to significant concerns about intellectual property infringement, cost, environmental impact and other issues.
- Some leading startups of generative AI's first wave are falling by the wayside. Last week, Inflection AI's leadership and top researchers decamped for Microsoft, while on Friday, Stability AI's CEO and co-founder resigned as the firm faced a talent and financial crunch.
A year ago, every board was pressuring its CEO to find ways to adopt generative AI as quickly as possible.
- Now, many are finding that even promising early experiments have proven tough to scale, and that what appear to be "good enough" results often aren't.
What they're saying: Gary Marcus, a scientist who penned a blog post last year titled "What if generative AI turned out to be a dud?" tells Axios that, outside of a few areas such as coding, companies have found generative AI isn't the panacea they once imagined.
- "Almost everybody seemed to come back with a report like, 'This is super cool, but I can't actually get it to work reliably enough to roll out to our customers,'" Marcus says.
AI ethics expert Rumman Chowdhury tells Axios that the challenges are numerous and significant.
- "No one wants to build a product on a model that makes things up," says Chowdhury, CEO and co-founder of AI consulting firm Humane Intelligence.
- And, even when such issues are addressed, Chowdhury says the technology remains a "party trick" unless and until work is done to mitigate bias and discrimination.
Yes, but: This isn't the end of the road for generative AI, by any means. Every major new technology — even, or especially, a world-changing one — goes through this phase.
- The "trough of disillusionment" was first named and defined by consulting firm Gartner in 1995 as part of its theory of hype cycles in tech.
- Usable speech recognition was famously always five years away from reality until it finally arrived, and today is remarkably good even under less-than-ideal conditions.
- VR has famously entered several troughs, and its entry into the mainstream remains an open question.
The other side: The continued enthusiasm despite the setbacks was palpable at Nvidia's GTC conference in Silicon Valley last week, according to Chetan Sharma, a longtime telecom industry consultant who was at the event.
- Some tasks, like customer service and employee training, are seeing meaningful improvement from today's generative AI, while there's an emerging consensus that benefits in other areas will require better models and more refined data sets.
- "I think we are in that kind of mushy phase," Sharma tells Axios.
Zoom in: OpenAI CEO Sam Altman has offered few details on the next generation of OpenAI's underlying GPT engine, but has hinted it will offer the same order of magnitude improvement in general reasoning that the jump from GPT-3 to GPT-4 brought.
- "The thing that matters most is just that it gets smarter," Altman told Axios in a January onstage interview at the World Economic Forum. "GPT-2 couldn't do very much. GPT-3 could do more. GPT-4 could do a lot more. GPT-5 will be able to do a lot lot more."
- Marcus remains skeptical. "It's easy to say, 'Oh, we're just a few months away,'" Marcus says. "I don't think that we are in this particular case. Not because I don't think AI is or AGI is possible, but just because [this] particular technology has a lot of problems."
My thought bubble: I remain pretty optimistic about the long-term future of generative AI, and think it will help with a lot of tasks, both creative and routine.
- But the underlying models need improvement, and they will also have to be refined on customized data sets.
- The companies creating the models will need to figure out how to compensate the people whose work is in those data sets.
- Applying generative AI to many problems will require developing better interfaces for human-AI collaboration.
- Adding all this will take additional time and money.
2. Exclusive: Educators divided over AI's impact
Illustration: Natalie Peeples/Axios
Educators are split on whether artificial intelligence will help or hinder their careers, according to results of a study by the AI Education Project, first shared with Axios.
Why it matters: Education has been touted as one of the areas that could benefit the most from AI, yet the advent of ChatGPT and other tools has also added fresh challenges for teachers already stretched thin.
By the numbers: The study, which surveyed more than 1,000 K-12 teachers and administrators, found about 47% of respondents believe AI won't significantly affect their employment opportunities.
- About 28% believe that AI will be a boon to their careers, while 25% believe the impact will be negative.
- Less-experienced educators were mostly likely to be fearful of AI's impact, with nearly two-thirds describing themselves as either "extremely fearful" or "somewhat fearful" of AI. While they were the most hesitant, their fears were shared by anywhere from 45% to 59% of those with more experience.
- The survey showed a strong desire for more AI-specific resources, with 80% seeking more professional development for teachers and 75% supporting a curriculum that builds students' AI literacy.
- Administrators were more likely to have used the technology than teachers, nearly half of whom had yet to try generative AI in either their personal or professional life.
The big picture: Education has emerged as an early AI flashpoint. Some schools were quick to ban students' use of ChatGPT and other tools, while other educators liken its arrival to that of the calculator, suggesting clear policies on acceptable use rather than blanket bans.
- Longer-term, some have argued the use of generative AI could help teachers better understand where students are getting stuck and offer more personalized education to all.
- Online education pioneer Sal Khan has been a big backer of this perspective, creating the Khanmigo chatbot within Khan Academy and aggressively adopting AI within his test schools.
Between the lines: Educators surveyed also see opportunities to harness AI for personalized learning, better lesson plans and improved assessment.
Yes, but: Those surveyed raised concerns around misinformation, cheating, equity and privacy.
3. Training data
- AI companies should submit to independent audits of high-risk AI systems, per an announcement by the NTIA today. (Axios Pro)
- Apple announced its annual WWDC developer event will take place June 10-14, where it is expected to detail generative AI features for iOS coming next fall. (The Verge)
- Online misinformation and conspiracy theories about the collapse of Baltimore's Francis Scott Key Bridge spread quickly on Elon Musk's X. (Axios)
4. + This
It's not an official Olympic event, but there was just a race in Paris that saw waiters and waitresses competing over who could successfully carry a croissant, coffee cup and glass of water fastest to the finish line.
Thanks to Scott Rosenberg and Megan Morrone for editing this newsletter and to Caitlin Wolper for copy editing it.
Sign up for Axios AI+

Scoops on the AI revolution and transformative tech, from Ina Fried, Madison Mills, Ashley Gold and Maria Curi.

