May 2, 2024 - Technology

Magical mystery chatbot is likely from OpenAI

headshot
Animated illustration of a pattern of speech bubbles made of search bars with blinking cursors.

Illustration: Shoshana Gordon/Axios

A powerful new chatbot that showed up briefly on a testing site this week, stirring buzz among AI experts and enthusiasts, is likely from OpenAI.

The big picture: New AI tools capable of performing as well as OpenAI's most advanced GPT-4 model are rare and cost a ton of money to build and train, which is why they usually come from one of the known big players.

  • A number of signs suggest that this mystery bot is OpenAI's — but it's more likely to be an improvement on an existing model than a brand new one.

Driving the news: The new "gpt-2 chatbot" showed up roughly a week ago on LMSYS, a well-known site that researchers use to test their products in a "chatbot arena."

  • It impressed many observers and sparked a speculative frenzy — and then, Tuesday, it was gone again.

OpenAI CEO Sam Altman fed the fire on Monday, posting on X: "i do have a soft spot for gpt2."

  • Speaking on Wednesday at Harvard University, Altman told an audience that the mystery bot is not GPT-4.5, what many see as the likely next major update to GPT-4.
  • An OpenAI representative declined to comment further.

Between the lines: There is more than one way to improve generative AI's state of the art. One way is to make large language models even larger by increasing their number of neuron-like parameters, making them more versatile and more costly.

  • Other ways include fine-tuning an existing model or giving it new features and capabilities.

What they're saying: Those who tried out the mystery chatbot while it was available found several indications within its responses that it was trained either on existing OpenAI models or similarly to them.

  • Some praised advances in reasoning and handling of math problems that have flummoxed earlier models from both OpenAI and rivals, such as Anthropic and Google.
  • LMSYS said in a post on X, "In line with our policy, we've worked with several model developers in the past to offer community access to unreleased models/checkpoints (e.g., mistral-next, gpt2-chatbot) for preview testing."
  • "Due to unexpectedly high traffic & capacity limit, we have to temporarily take gpt2-chatbot offline. Please stay-tuned for its broader releases :)"
  • University of Pennsylvania professor Ethan Mollick also posted on X about the bot, "No one knows who made it or what it is, but I have been playing with it a little and it appears to be in the same rough ability level as GPT-4."
  • "A mysterious GPT-4 class model? Neat!" Mollick added.
  • "I was skeptical about the GPT2 chatbot, but it is undoubtedly more capable than open-source models and, in some cases, better than GPT4-turbo," Neural Love CEO Denis Shiryaev said on X.

Our thought bubble: If "gpt2-chatbot" does turn out to be an OpenAI project, the company may have chosen this stealth mode to drum up excitement or to see how the bot performs in the wild.

  • Whether it's a test or a tease, it's likely we'll know more soon.

Zoom out: The AI industry expects to shift this year toward agents, allowing AI models to act on a user's behalf, rather than just return information.

  • Such a shift could dramatically increase the usefulness of chatbots, but also creates fresh risk, especially without improvements in reasoning and a reduction of hallucinations.
Go deeper