Axios AI+

June 24, 2024
Sending good luck to the Edmonton Oilers in their Game 7 match against the Florida Panthers. I've been rooting for them since I was a kid, and it's been 34 years since they last hoisted the Stanley Cup.
Situational awareness: The EU charged Apple with violating Europe's new digital competition laws. At issue: Apple's rules blocking companies from "steering" customers outside the App Store to avoid Apple fees.
Today's AI+ is 1,234 words, a 4.5-minute read.
1 big thing: AI's Achilles heel
Generative AI makes things up. It can't distinguish between fact and fiction. It asserts its fabrications with confident authority.
State of play: All that was true in 2022 when ChatGPT debuted. It's still true today. But the tech industry keeps remodeling the entire digital universe around AI as if none of it were happening.
Why it matters: GenAI's unreliability may just be a nuisance when you're asking it for a recipe or a video recommendation.
- It's far more troubling when the technology moves into medicine, finance, law and other realms where "oops, sorry" doesn't cut it.
Driving the news: Perplexity, the popular AI "answer engine," readily spouts inaccuracies and garbled or uncredited rewrites of published material, both Forbes and Wired have recently reported.
- Wired called Perplexity a "bulls--t machine."
- Perplexity's rough patch follows broader disappointment among some users with Google's AI-generated search-result summaries.
Fun fact: Just days after ChatGPT's release, computer scientists Arvind Narayanan and Sayash Kapoor declared, "ChatGPT is a bulls--t generator." The same concept has now inspired a research paper titled "ChatGPT is bulls--t."
- "Because these programs cannot themselves be concerned with truth, and because they are designed to produce text that looks truth-apt without any actual concern for truth, it seems appropriate to call their outputs bulls--t," the paper's University of Glasgow authors write.
Between the lines: GenAI's "hallucinations," "confabulations" and errors aren't random bugs — they're a fundamental part of how this technology works.
- Every time they open their hypothetical mouths, large language models like Google's Gemini and OpenAI's GPT are literally guessing the next word.
- They don't "know" anything — they're just calculating what the best way to continue a sentence might be.
Model builders can adjust the level of randomness ("temperature") to make a chatbot wilder and more inventive or dull but more reliable.
- Users expect AI to behave like any traditional computing tool — with consistency and logic — whereas genAI always has an element of unpredictability and randomness.
Zoom out: People outside the AI bubble keep hearing that AI is a superpower force that will answer their questions, diagnose their illnesses and/or steal their jobs — so the technology's fallibility, and the inevitability of that fallibility, comes as a shock.
There are three ways the tech industry could tackle this problem.
1. More accuracy. AI makers could try to make their tools more trustworthy with better data and training. Models could be programmed to stop trying so hard to "fill in the blanks" with made-up answers.
- Yes, but: This is harder than it sounds, and there's no guarantee of success. Also, AI that's tuned to be less eager to help is going to feel less useful. Nonetheless, it's the strategy most companies are pursuing today.
2. Less confidence. AI makers could modify their chatbots to behave with less assurance and admit when they're just not sure of an answer.
- Yes, but: The AI would have to be able to tell when it didn't know an answer, and even that is proving an elusive goal.
3. Embrace BS. AI makers could declare that making things up is a feature rather than a bug — and reposition their tools as goads to creativity rather than oracles of universal knowledge.
- Yes, but: This would require accepting a much smaller AI market than widely expected now. That's unpalatable for investors who've already plowed in billions and expect monster returns.
The bottom line: It's hard to sell a new technology as world-changing and civilization-saving (or species-threatening) when you can't explain how it arrives at any particular output and can't promise that it's not going to keep going off the rails.
- Silicon Valley has made this uncomfortable bed for itself. AI makers will keep trying to lie in it.
2. "Trust the trendlines": 10 takeaways from a hot paper
A 50,000-word, 165-page essay that's making the rounds in Silicon Valley predicts accelerating advances in AI — and counters recent stirrings of disillusionment in the AI industry's work.
- "Situational Awareness: The Decade Ahead" is the work of Leopold Aschenbrenner. Formerly of OpenAI's Superalignment team, he's now founder of an investment firm focused on artificial general intelligence (AGI).
Here are 10 takeaways:
1. "Trust the trendlines ... The trendlines are intense, and they were right."
- "The magic of deep learning is that it just works — and the trendlines have been astonishingly consistent, despite naysayers at every turn."
2. "Over and over again, year after year, skeptics have claimed 'deep learning won't be able to do X' and have been quickly proven wrong."
- "If there's one lesson we've learned from the past decade of AI, it's that you should never bet against deep learning."
3. It's "strikingly plausible that by 2027, models will be able to do the work of an AI researcher/engineer."
4. "By 2027, rather than a chatbot, you're going to have something that looks more like an agent, like a coworker."
5. The data wall: "We're running out of internet data. That could mean that, very soon, the naive approach to pretraining larger language models on more scraped data could start hitting serious bottlenecks."
6. "AI progress won't stop at human-level … We would rapidly go from human-level to vastly superhuman AI systems."
7. AI products are likely to become "the biggest revenue driver for America's largest corporations, and by far their biggest area of growth. Forecasts of overall revenue growth for these companies would skyrocket."
- "Stock markets would follow; we might see our first $10T company soon thereafter. Big tech at this point would be willing to go all out, each investing many hundreds of billions (at least) into further AI scaleout. We probably [will] see our first many-hundred-billion-dollar corporate bond sale."
8. "Our failure today" to erect sufficient barriers around research on artificial general intelligence "will be irreversible soon: in the next 12-24 months, we will leak key AGI breakthroughs to the [Chinese Communist Party]. It will be the national security establishment's single greatest regret before the decade is out."
9. Superintelligence "will be the United States' most important national defense project."
10. There's "no crack team coming to handle this. ... Right now, there's perhaps a few hundred people in the world who realize what's about to hit us, who understand just how crazy things are about to get, who have situational awareness."
Reality check: Aschenbrenner, who has roots in the effective altruism movement, is an AI investor. So he's not a disinterested party.
- His belief that AGI will inevitably evolve out of today's genAI language models remains hotly contested in the industry. Right now, the biggest players are more focused on proving the value and utility of today's AI than forced-marching toward superintelligence.
3. Training data
- Add Meta to the list of companies (including Google, Anthropic and Perplexity) that Apple is reportedly in talks with about bringing their AI to the iPhone. Apple already announced a deal with OpenAI. (The Wall Street Journal)
- Apple says it won't release AI features in Europe due to regulatory concerns. (Axios)
- Amazon is reportedly considering charging between $5-$10 for a monthly subscription to its Alexa genAI features. (Reuters)
- OpenAI acquired an enterprise search and analytics startup, its first purchase and integration of another company. (Bloomberg)
- Stability AI got a new CEO and a fresh infusion of cash from investors, including Sean Parker. (The Information)
4. + This
Not only did I learn that there used to be coin-operated typewriters, but I read that Ray Bradbury used 98 dimes to write a draft of what became "Fahrenheit 451."
Thanks to Megan Morrone and Scott Rosenberg for editing this newsletter and to Caitlin Wolper for copy editing it.
Sign up for Axios AI+





