Generative AI is also a revolution for computer interfaces
Add Axios as your preferred source to
see more of our stories on Google.
Illustration: Sarah Grillo/Axios
Every revolution in tech starts with a better way for normal humans to access computing power — from punchcards and command lines to windows and a mouse to smartphones and touchscreens. AI is poised to be the next leap.
Why it matters: One big reason the tech world is so giddy over generative AI is that industry veterans see ChatGPT's conversational fluency as a key that can unlock the next level in this game of endowing everyday users with digital powers.
- Think of the hours saved — and, in theory, productivity gained — if you can simply tell your chatbot to clean up your inbox, change your system settings or connect to a printer.
- The upside is, you won't have to know how to do those things yourself. That's a downside, too: It means individual users may end up with less skill and less direct control.
Driving the news: A number of big players are already using the power of large language models to make chat-style interaction a key way to interact with their software.
- Microsoft this week unveiled a Windows Copilot which adds a side pane where users can both summon the AI-powered Bing search engine and change a whole host of settings that, until now, required knowing how to dive into an arcane system of control panels.
- "I think over the coming years, this will become an expectation for how all software works," CTO Kevin Scott said in a statement.
- Microsoft is also adopting the same standard for plug-ins that OpenAI is using for ChatGPT, meaning that the chatbot isn't just a user interface, it's also a new platform for developers to create apps — and a new front end for controlling other software.
- Adobe announced a new generative fill tool for Photoshop that lets people describe what they want to take place in an area, say put a mountain here or remove this object. The company sees big potential in using such descriptive commands to take all manner of actions.
What they're saying: Windows head Panos Panay told Axios that AI is bringing about a "generational shift" in computer interfaces.
- "It will feel magical," Panay said in an interview.
Yes, but: It's going to take time for the full power and nuance of AI-as-interface to emerge, Panay said, pointing out that it took some years after the mouse before we got to a scroll wheel.
- "It's a migration; this isn’t an overnight change," he said. "You learned to point and then you learned to scroll," he said.
- Panay described a future in which queries made in the chat-based Copilot can stay there or, as needed, move into apps in the main Windows interface. "We will put it where you want it," he said.
- Some of the computing work making all this AI-based chat happen will continue to happen in the cloud, but for privacy and other reasons, some work will shift back to the device, he said — noting that there are 200 million PCs out there with enough computing power to run some AI models locally.
Be smart: Using a chatbot to interact with software doesn't always mean you are seeing generative AI in action. Sometimes it's just about using natural language to say, change the settings or invoke a series of commands with a single instruction.
The other side: Magical as it often feels, the chat interface comes with its own drawbacks.
- The empty chat window is something of a cipher. It doesn't provide users with much indication of what it can do or how to use it; there's no easy way to poke around and get a sense of its capabilities.
Flashback: Tech companies have talked for years about natural language interfaces. Back in the late 1990s, Clippy used to pop up in Word when it looked like you were writing a letter or resume and might need help.
- But Clippy's abilities were constrained by the limits of computers to understand all the different ways that humans describe the actions they want to take.
- It's only with the arrival of large language models trained on all these ways that computers have reached a point they seem poised to speak our language, rather than humans having to learn to "speak computer."
- Perhaps as a sign of things to come, Microsoft brought Clippy back as an emoji in Windows 11, following years of exile.
What's next: Expect to see chatbot-like interfaces show up all over the place. Once the tech industry gloms on to a trend, it tends to happen everywhere, for better and worse.
- For example, once smartphones ushered in an app-centered world, every company and organization felt obligated for a while to create their own app — even if it was just for a conference you were attending once.
