Oct 15, 2019

Machine learning can't flag false news, new studies show

Current machine learning models aren't yet up to the task of distinguishing false news reports, two new papers by MIT researchers show.

The big picture: After different researchers showed that computers can convincingly generate made-up news stories without much human oversight, some experts hoped that the same machine-learning-based systems could be trained to detect such stories. But MIT doctoral student Tal Schuster's studies show that, while machines are great at detecting machine-generated text, they can't identify whether stories are true or false.

Details: Many automated fact-checking systems are trained using a database of true statements called Fact Extraction and Verification (FEVER).

  • In one study, Schuster and team showed that machine learning-taught fact-checking systems struggled to handle negative statements ("Greg never said his car wasn't blue") even when they would know the positive statement was true ("Greg says his car is blue").
  • The problem, say the researchers, is that the database is filled with human bias. The people who created FEVER tended to write their false entries as negative statements and their true statements as positive statements — so the computers learned to rate sentences with negative statements as false.
  • That means the systems were solving a much easier problem than detecting fake news. "If you create for yourself an easy target, you can win at that target," said MIT professor Regina Barzilay. "But it still doesn't bring you any closer to separating fake news from real news."
  • Both studies were headed by Schuster with teams of MIT collaborators.

The bottom line: The second study showed that machine-learning systems do a good job detecting stories that were machine-written, but not at separating the true ones from the false ones.

Yes, but: While you can generate bogus news stories more efficiently using automated text, not all stories created by automated processes are untrue.

Go deeper

Facebook employees stage "virtual walkout"

Illustration: Sarah Grillo/Axios

"Dozens" of Facebook employees staged a "virtual walkout" Monday over the company's decision not to take action against President Trump's provocative messages in the face of nationwide protests against police violence, the New York Times reports.

Why it matters: While Twitter added fact-check labels and hid the president's most inflammatory tweet — "when the looting starts, the shooting starts" — Facebook has said Trump's statements do not violate its policies, and that the platform aims to promote free speech.

This story is developing. Please check back for updates.

Updated 13 mins ago - Politics & Policy

Trump lashes out at governors, urges them to "dominate" protesters

President Trump berated the nation’s governors in a video teleconference call Monday, calling many of them "weak" and demanding tougher crackdowns on the protests that erupted throughout the country following the killing of George Floyd, according to multiple reports.

The big picture: Trump blamed violence on the "the radical left" and told the governors, who were joined by law enforcement and national security officials, that they have to "dominate" protesters and "arrest people" in order to bring an end to the unrest.

Updated 13 mins ago - Politics & Policy

Coronavirus dashboard

Photo Illustration: Sarah Grillo/Axios. Photos: Noam Galai, Jamie McCarthy, Josep Lago/AFP, Alfredo Estrella/AFP, and Narayan Maharjan/NurPhoto, all via Getty Images

  1. Global: Total confirmed cases as of 12:30 p.m. ET: 6,206,773 — Total deaths: 372,752 — Total recoveries — 2,661,643Map.
  2. U.S.: Total confirmed cases as of 12:30 p.m. ET: 1,793,780 — Total deaths: 104,3450 — Total recoveries: 444,758 — Total tested: 16,936,891Map.
  3. Public health: Fauci says“My meetings with the president have been dramatically decreased” — Coronavirus looms over George Floyd protests across the country.
  4. Climate: Your guide to comparing climate change and coronavirus.
  5. Economy: A busy week for IPOs despite upheaval from protests and pandemic.
  6. World: Former FDA commissioner says "this is not the time" to cut ties with WHO.