Current machine learning models aren't yet up to the task of distinguishing false news reports, two new papers by MIT researchers show.

The big picture: After different researchers showed that computers can convincingly generate made-up news stories without much human oversight, some experts hoped that the same machine-learning-based systems could be trained to detect such stories. But MIT doctoral student Tal Schuster's studies show that, while machines are great at detecting machine-generated text, they can't identify whether stories are true or false.

Details: Many automated fact-checking systems are trained using a database of true statements called Fact Extraction and Verification (FEVER).

  • In one study, Schuster and team showed that machine learning-taught fact-checking systems struggled to handle negative statements ("Greg never said his car wasn't blue") even when they would know the positive statement was true ("Greg says his car is blue").
  • The problem, say the researchers, is that the database is filled with human bias. The people who created FEVER tended to write their false entries as negative statements and their true statements as positive statements — so the computers learned to rate sentences with negative statements as false.
  • That means the systems were solving a much easier problem than detecting fake news. "If you create for yourself an easy target, you can win at that target," said MIT professor Regina Barzilay. "But it still doesn't bring you any closer to separating fake news from real news."
  • Both studies were headed by Schuster with teams of MIT collaborators.

The bottom line: The second study showed that machine-learning systems do a good job detecting stories that were machine-written, but not at separating the true ones from the false ones.

Yes, but: While you can generate bogus news stories more efficiently using automated text, not all stories created by automated processes are untrue.

Go deeper

Trump tightens screws on ByteDance to sell Tiktok

Illustration: Aïda Amer/Axios

President Trump added more pressure Friday night on China-based TikTok parent ByteDance to exit the U.S., ordering it to divest all assets related to the U.S. operation of TikTok within 90 days.

Between the lines: The order means ByteDance must be wholly disentangled from TikTok in the U.S. by November. Trump had previously ordered TikTok banned if ByteDance hadn't struck a deal within 45 days. The new order likely means ByteDance has just another 45 days after that to fully close the deal, one White House source told Axios.

Updated 1 hour ago - Politics & Policy

Coronavirus dashboard

Illustration: Aïda Amer/Axios

  1. Global: Total confirmed cases as of 9:30 p.m. ET: 21,056,850 — Total deaths: 762,293— Total recoveries: 13,100,902Map.
  2. U.S.: Total confirmed cases as of 9:30 p.m ET: 5,306,215 — Total deaths: 168,334 — Total recoveries: 1,796,309 — Total tests: 65,676,624Map.
  3. Health: CDC: Survivors of COVID-19 have up to three months of immunity Fauci believes normalcy will return by "the end of 2021" with vaccine — The pandemic's toll on mental health — FDA releases first-ever list of medical supplies in shortage.
  4. States: California passes 600,000 confirmed coronavirus cases.
  5. Cities: Coronavirus pandemic dims NYC's annual 9/11 Tribute in Light.
  6. Business: How small businesses got stiffed — Unemployment starts moving in the right direction.
  7. Politics: Biden signals fall strategy with new ads.

Harris: "Women are going to be a priority" in Biden administration

Sen. Kamala Harris at an event in Wilmington, Del. Photo: Drew Angerer/Getty Images

In her first sit-down interview since being named Joe Biden's running mate, Sen. Kamala Harris talked about what she'll do to fight for women if elected VP, and how the Democrats are thinking about voter turnout strategies ahead of November.

What they're saying: "In a Biden-Harris administration women are going to be a priority, understanding that women have many priorities and all of them must be acknowledged," Harris told The 19th*'s Errin Haines-Whack.