Magical mystery chatbot is likely from OpenAI
Add Axios as your preferred source to
see more of our stories on Google.
/2024/05/01/1714594326549.gif?w=3840)
Illustration: Shoshana Gordon/Axios
A powerful new chatbot that showed up briefly on a testing site this week, stirring buzz among AI experts and enthusiasts, is likely from OpenAI.
The big picture: New AI tools capable of performing as well as OpenAI's most advanced GPT-4 model are rare and cost a ton of money to build and train, which is why they usually come from one of the known big players.
- A number of signs suggest that this mystery bot is OpenAI's — but it's more likely to be an improvement on an existing model than a brand new one.
Driving the news: The new "gpt-2 chatbot" showed up roughly a week ago on LMSYS, a well-known site that researchers use to test their products in a "chatbot arena."
- It impressed many observers and sparked a speculative frenzy — and then, Tuesday, it was gone again.
OpenAI CEO Sam Altman fed the fire on Monday, posting on X: "i do have a soft spot for gpt2."
- Speaking on Wednesday at Harvard University, Altman told an audience that the mystery bot is not GPT-4.5, what many see as the likely next major update to GPT-4.
- An OpenAI representative declined to comment further.
Between the lines: There is more than one way to improve generative AI's state of the art. One way is to make large language models even larger by increasing their number of neuron-like parameters, making them more versatile and more costly.
- Other ways include fine-tuning an existing model or giving it new features and capabilities.
What they're saying: Those who tried out the mystery chatbot while it was available found several indications within its responses that it was trained either on existing OpenAI models or similarly to them.
- Some praised advances in reasoning and handling of math problems that have flummoxed earlier models from both OpenAI and rivals, such as Anthropic and Google.
- LMSYS said in a post on X, "In line with our policy, we've worked with several model developers in the past to offer community access to unreleased models/checkpoints (e.g., mistral-next, gpt2-chatbot) for preview testing."
- "Due to unexpectedly high traffic & capacity limit, we have to temporarily take gpt2-chatbot offline. Please stay-tuned for its broader releases :)"
- University of Pennsylvania professor Ethan Mollick also posted on X about the bot, "No one knows who made it or what it is, but I have been playing with it a little and it appears to be in the same rough ability level as GPT-4."
- "A mysterious GPT-4 class model? Neat!" Mollick added.
- "I was skeptical about the GPT2 chatbot, but it is undoubtedly more capable than open-source models and, in some cases, better than GPT4-turbo," Neural Love CEO Denis Shiryaev said on X.
Our thought bubble: If "gpt2-chatbot" does turn out to be an OpenAI project, the company may have chosen this stealth mode to drum up excitement or to see how the bot performs in the wild.
- Whether it's a test or a tease, it's likely we'll know more soon.
Zoom out: The AI industry expects to shift this year toward agents, allowing AI models to act on a user's behalf, rather than just return information.
- Such a shift could dramatically increase the usefulness of chatbots, but also creates fresh risk, especially without improvements in reasoning and a reduction of hallucinations.
