The chatbot wars get personal
Add Axios as your preferred source to
see more of our stories on Google.
Illustration: Brendan Lynch/Axios
Google's Tuesday announcement of several new Gemini features is part of a broader effort by tech giants to make their chatbots more personal by giving them access to more of your data.
Why it matters: Apple and Microsoft have also taken steps to combine the generic knowledge of a large language model with a user's personal data — a move that makes AI assistants both more practical and more of a potential privacy problem.
The big picture: The first generative AI chatbots knew a lot about the world, but almost nothing about the specific person using them.
- The race to combine world knowledge and personal information is now on — though it's still in its earliest stages.
Driving the news: Google is making several moves to allow Gemini to access users' information.
- One of the new options is what Google describes as a "contextual overlay" that allows Android owners to press a button and have Gemini try to answer questions based on whatever is on the phone's screen.
- Pixel Screenshots — which is only available on the new Pixel 9 series of phones, announced on Tuesday — allows people to store specific screenshots so they can later ask Gemini for the information on them, like the door code for an Airbnb reservation.
- All the work is done on the device, Google said, meaning users' personal information isn't sent up to cloud servers. That's part of why the phones have added built-in memory to run the most powerful current AI models. Google says it doesn't have access to any of the information generated by Pixel Screenshots.
- Google is also offering new extensions that allow Gemini to bring in data from other Google products, including Calendar, Keep, Tasks and YouTube Music.
- There's also a "call notes" feature that saves summaries of phone calls.
What they're saying: "We feel like we're at the beginning of something that's really, really exciting," Jenny Blackburn, Google's VP of Gemini user experience, told Axios.
Yes, but: Blackburn acknowledges that generative AI still has flaws and says that, in part, is why the assistant always asks a user to confirm their intent before taking action.
- "The helpfulness that people get from Gemini outweighs the challenges," she said.
Zoom out: Google's approach to screenshots, especially Pixel Screenshots, is reminiscent of what Microsoft is aiming to do with Recall, the signature feature of its new generation of computers, known as Copilot+ PCs.
- Like the new Gemini feature, Recall relies on screenshots of what users are doing.
Microsoft's plan, unlike Google's, is to scrape everything a user sees and store it over time.
- After a blowback, Microsoft delayed the Recall feature and said it would be turned off by default, among other changes. Microsoft also has said people will be able to choose whether a particular app or website is captured.
Apple's efforts with Apple Intelligence rely on even more personal data, but the company says it has taken extensive privacy measures to ensure that the information is only used in processing each particular request.
- The company is also rolling out its AI features over the next year, more slowly than its competitors — and has said it won't bring them to Europe at all amid a lack of regulatory clarity.
Meta is something of a wild card here, as it has a ton of personal information on its users, but has been less clear about what it will and won't do going forward.
- It has said it plans to use publicly shared data from Instagram and its other services to help train its models, a move that has prompted concern from European regulators.
- In response, Meta has said it won't be offering its future multimodal AI models in Europe.
