Publisher retracts computer-generated research papers

A publisher retracted several scientific papers after it was discovered they had been generated by a computer.

Why it matters: Computer-produced fraudulent papers have been a problem in academia for years, and as AI text-generation capabilities improve, it may become increasingly difficult to tell the real from the fake.

Driving the news: The blog Retraction Watch reported today that IOP Publishing had retracted five papers published between 2018 and 2020 in the company's Conference Series: Earth and Environmental Science.

• The papers showed clear signs of being generated by SciGen, an "automatic CS [computer science] paper generator" developed in 2005, or another similar system.

Of note: One of the fake papers was titled "Neural Networks Considered Harmful," which, even if the paper wasn't real, would seem to prove its point.

Background: SciGen originated as a kind of MIT prank. Three grad students wanted to expose how scientific conferences demanded paid papers by researchers, but did very little to actually check them.

• SciGen was a simple system that spewed out random gobbledygook, but it was reasonably academic-sounding random gobbledygook, and so the fake paper was accepted.
• In the years following, scientists identified scores of SciGen-created fake papers that were accepted by legitimate publishers.

Be smart: Right now, the continued publication of computer-generated papers says less about the writing abilities of the computers themselves and more about the standards of some scientific publishers — especially since software to detect SciGen-created papers has existed for years.

• But as AI text-generation systems like GPT-3 grow more sophisticated, such fraud will likely become harder to spot — and more enticing to attempt.

The bottom line: This article, at least, is currently 100% human-generated.

• Though that's probably what a computer would say.