Computers that learn words from texts written by humans capture their meaning but also our biases, a new study shows.

Why it matters: Machine learning is being eyed to sift through resumes in an effort to reduce discrimination in hiring, analyze loan applications and to predict criminal behavior while reducing racial profiling. The unintended biases found in artificial intelligence raise ethical questions about whether and how to deploy the technology without reinforcing stereotypes. (See Exhibit A, the racist Microsoft bot.)

How it works: The researchers created a test for how closely the AI associates different words and uncovered gender and racial biases similar to those of humans that are well-known from psychological studies. They found that European-American names were more closely associated with pleasant words (honest, gentle, happy) whereas unpleasant words (divorce, filth, jail) were more likely to be attributed to African-American names. Young people were considered pleasant, old people were not. They then looked at gender bias and found the AI associated women more so than men with family and the arts than with mathematics.

Thought bubble: Context provides bias but also meaning. How much bias can be removed before that meaning is lost?

The study authors don't recommend untraining the machine because of the risk of removing crucial knowledge about the world. "Artificial intelligence learns biases but it needs the awareness not to make prejudiced decisions. Since machines do possess self-awareness the way humans do, a human in the loop can help machines make ethical decisions," says Princeton's Aylin Caliskan.

Go deeper

8 hours ago - World

China-Iran deal envisions massive investments from Beijing

Illustration: Aïda Amer/Axios

China and Iran have negotiated a deal that would see massive investments flow into Iran, oil flow out, and collaboration increase on defense and intelligence.

Why it matters: If the proposals become reality, Chinese cash, telecom infrastructure, railways and ports could offer new life to Iran’s sanctions-choked economy — or, critics fear, leave it inescapably beholden to Beijing.

Updated 9 hours ago - Politics & Policy

Coronavirus dashboard

Illustration: Aïda Amer/Axios

  1. Global: Total confirmed cases as of 7 p.m. ET: 13,048,249 — Total deaths: 571,685 — Total recoveries — 7,215,865Map.
  2. U.S.: Total confirmed cases as of 7 p.m. ET: 3,353,348— Total deaths: 135,524 — Total recoveries: 1,031,856 — Total tested: 40,282,176Map.
  3. World: WHO head: There will be no return to the "old normal" for foreseeable future — Hong Kong Disneyland closing due to surge.
  4. States: Houston mayor calls for two-week shutdownCalifornia orders sweeping rollback of open businesses — Cuomo says New York will use formula to determine if reopening schools is safe.
  5. Education: Los Angeles schools' move to online learning could be a nationwide tipping point.

House Judiciary Committee releases transcript of Geoffrey Berman testimony

Geoffrey Berman. Photo: Alex Wong/Getty Images

The House Judiciary Committee on Monday released the transcript of its closed-door interview with Geoffrey Berman, the former top federal prosecutor in Manhattan who was forced out by Attorney General Bill Barr last month.

Why it matters: House Democrats have seized on Berman's testimony, in which he claimed the attorney general sought to "entice" him into resigning so that he could be replaced by SEC chairman Jay Clayton, to bolster allegations that the Justice Department has been politicized under Barr.