Current machine learning models aren't yet up to the task of distinguishing false news reports, two new papers by MIT researchers show.

The big picture: After different researchers showed that computers can convincingly generate made-up news stories without much human oversight, some experts hoped that the same machine-learning-based systems could be trained to detect such stories. But MIT doctoral student Tal Schuster's studies show that, while machines are great at detecting machine-generated text, they can't identify whether stories are true or false.

Details: Many automated fact-checking systems are trained using a database of true statements called Fact Extraction and Verification (FEVER).

  • In one study, Schuster and team showed that machine learning-taught fact-checking systems struggled to handle negative statements ("Greg never said his car wasn't blue") even when they would know the positive statement was true ("Greg says his car is blue").
  • The problem, say the researchers, is that the database is filled with human bias. The people who created FEVER tended to write their false entries as negative statements and their true statements as positive statements — so the computers learned to rate sentences with negative statements as false.
  • That means the systems were solving a much easier problem than detecting fake news. "If you create for yourself an easy target, you can win at that target," said MIT professor Regina Barzilay. "But it still doesn't bring you any closer to separating fake news from real news."
  • Both studies were headed by Schuster with teams of MIT collaborators.

The bottom line: The second study showed that machine-learning systems do a good job detecting stories that were machine-written, but not at separating the true ones from the false ones.

Yes, but: While you can generate bogus news stories more efficiently using automated text, not all stories created by automated processes are untrue.

Go deeper

Dave Lawler, author of World
18 mins ago - World

Special report: Trump's hopes of nuclear deal with Putin come down to the wire

Illustration: Lazaro Gamio/Axios

A surprise offer from Vladimir Putin has the U.S. and Russia once again circling a potential pre-election nuclear deal.

The big picture: The last treaty constraining the U.S. and Russia, New START, is due to expire on Feb. 5, 2021, two weeks after the next U.S. presidential inauguration. For the first time since the height of the Cold War, the nuclear guardrails could come off.

The cliffhanger could be ... Georgia

Illustration: Sarah Grillo/Axios

It hasn't backed a Democrat for president since 1992, but Georgia's changing demographics may prove pivotal this year — not only to Trump v. Biden, but also to whether Democrats take control of the Senate.

Why it matters: If the fate of the Senate did hinge on Georgia, it might be January before we know the outcome. Meanwhile, voters' understanding of this power in the final days of the election could juice turnout enough to impact presidential results.

Amy Harder, author of Generate
6 hours ago - Energy & Environment

Climate change goes mainstream in presidential debate

Photo illustration: Aïda Amer/Axios. Photo: Olivier Douliery-Pool/Getty

The most notable part of Thursday’s presidential debate on climate change was the fact it was included as a topic and assumed as a fact.

The big picture: This is the first time in U.S. presidential history that climate change was a featured issue at a debate. It signals how the problem has become part of the fabric of our society. More extreme weather, like the wildfires ravaging Colorado, is pushing the topic to the front-burner.