Current machine learning models aren't yet up to the task of distinguishing false news reports, two new papers by MIT researchers show.

The big picture: After different researchers showed that computers can convincingly generate made-up news stories without much human oversight, some experts hoped that the same machine-learning-based systems could be trained to detect such stories. But MIT doctoral student Tal Schuster's studies show that, while machines are great at detecting machine-generated text, they can't identify whether stories are true or false.

Details: Many automated fact-checking systems are trained using a database of true statements called Fact Extraction and Verification (FEVER).

  • In one study, Schuster and team showed that machine learning-taught fact-checking systems struggled to handle negative statements ("Greg never said his car wasn't blue") even when they would know the positive statement was true ("Greg says his car is blue").
  • The problem, say the researchers, is that the database is filled with human bias. The people who created FEVER tended to write their false entries as negative statements and their true statements as positive statements — so the computers learned to rate sentences with negative statements as false.
  • That means the systems were solving a much easier problem than detecting fake news. "If you create for yourself an easy target, you can win at that target," said MIT professor Regina Barzilay. "But it still doesn't bring you any closer to separating fake news from real news."
  • Both studies were headed by Schuster with teams of MIT collaborators.

The bottom line: The second study showed that machine-learning systems do a good job detecting stories that were machine-written, but not at separating the true ones from the false ones.

Yes, but: While you can generate bogus news stories more efficiently using automated text, not all stories created by automated processes are untrue.

Go deeper

Updated 55 mins ago - Politics & Policy

Coronavirus dashboard

Illustration: Sarah Grillo/Axios

  1. Global: Total confirmed cases as of 10 a.m. ET: 19,412,292 — Total deaths: 722,066 — Total recoveries — 11,773,112Map.
  2. U.S.: Total confirmed cases as of 10 a.m. ET: 4,945,795 — Total deaths: 161,456 — Total recoveries: 1,623,870 — Total tests: 60,415,558Map.
  3. Politics: Trump says he's prepared to sign executive orders on coronavirus aid.
  4. Public health: Fauci says chances are "not great" that COVID-19 vaccine will be 98% effective
  5. Science: Indoor air is the next coronavirus frontline.
  6. World: Africa records over 1 million coronavirus cases — Gates Foundation puts $150 million behind coronavirus vaccine production.
59 mins ago - Health

Indoor air is the next coronavirus frontline

Illustration: Eniola Odetunde/Axios

A growing body of research has made it clear that airborne transmission of the coronavirus is possible.

Why it matters: That fact means indoor spaces can become hot spots. Those spaces also happen to be where most business and schooling takes place, so any hope for a return to normality will require better ways of filtering indoor air.

The silver linings of online school

Illustration: Annelise Capossela/Axios

Online learning can be frustrating for students, teachers and parents, but some methods are working.

The big picture: Just as companies are using this era of telework to try new things, some principals, teachers and education startups are treating remote learning as a period of experimentation, too.