ChatGPT sparks surge of AI detection tools
Add Axios as your preferred source to
see more of our stories on Google.
Illustration: Aïda Amer/Axios
The rise of generative AI tools is creating parallel demand for a new class of systems that can help distinguish AI-generated text and images from those created by humans.
Why it matters: Educators, in particular, are concerned about students turning in work created by an AI system. But experts are also worried about how generative AI can create a flood of misinformation and impersonation. Detection tools, if they can be made sufficiently accurate, could help.
Driving the news: OpenAI released a free web-based tool Tuesday designed to help distinguish AI-generated text from that written by humans. But it isn't alone in trying to address this issue. A number of startups, organizations and individuals have also released or are developing AI-detection software.
- GPTZero was created by college student Edward Tian and works similarly to the new tool from OpenAI, assessing the overall likelihood that a piece of content was machine-generated.
- Self-funded startup Fictitious.ai is testing a product with educational institutions that analyzes content paragraph-by-paragraph rather than just providing an overall score.
- Writer.com has a tool to help writers detect if their writing will pass as human, since search engines penalize content they believe to be machine-generated.
Yes, but: While detection tools show potential, the current tools are prone to making wrong calls, particularly when used on a writing sample that's very different from the ones used to train the system.
- The tools will no doubt improve, but so will the means for evading detection, likely creating something of a cat-and-mouse game.
- "We believe that techniques for detecting AI content will improve, and continue to surpass human ability to discern between machine- and human-written content," a Fictitious.ai representative told Axios. "With that said, there is clearly a limit at which sufficient human editing will always obscure traces of AI-generated content."
- OpenAI and others are also exploring other approaches, such as watermarking their systems' creations to make it easier to spot machine-generated content.
The big picture: Generative AI has hit a tipping-point of popularity, and that means all manner of computer-created content is about to explode.
- The challenge will be developing enough systems — both human and automated — to deal with the flood of additional content at a time when misinformation is already a significant problem.
In action: Here's how several detectors performed when we asked it to analyze a portion of yesterday's Login newsletter.
- OpenAI: "The classifier considers the text to be very unlikely AI-generated."
- GPT Zero: "Your text is most likely human written but there are some sentences with low perplexities."
- Fictitious.AI: This system broke the newsletter items down paragraph by paragraph, correctly discerning that all but the last part I submitted had less than a 5% chance of being AI-generated. For some reason, though, it decided the last line of a story about TikTok had an 85% chance of being AI-written. (It was us.) Line in question: "Don't be surprised to see such talks eventually resume, given how much there is at stake for the company and its investors."
- Writer.com: "Fantastic!" (I.e., we passed as human.)
And here's how the services evaluated a section of a ChatGPT-generated essay on the history of baseball cards. (It was a pretty good history, speaking as someone who knows a bit about the gum-infused cardboard collectibles.)
- OpenAI: "The classifier considers the text to be unclear if it is AI-generated."
- GPT Zero: "Your text is likely to be written entirely by AI."
- Fictitious.AI: The service categorized each paragraph as at least 88% likely to have been AI written with some listed as high as 98%.
- Writer.com: "You should edit your text until there’s less detectable AI content."
