Jul 29, 2018

AI stumbles in the spotlight

Illustration: Lazaro Gamio/Axios

Artificial intelligence experts — concerned about reported blunders with high-stakes AI systems from makers like Amazon and IBM — are urging more oversight, testing, and perhaps a fundamental rethinking of the underlying technology.

Why it matters: Wall Street, the military, and other sectors expect AI to make increasingly weighty decisions in the future — with less and less human involvement. But if the systems behave inaccurately or display biases, the consequences outside the lab could cause harm to real people.

In reports this week:

  • Amazon’s face-recognition platform, Rekognition, matched 28 members of Congress with mugshots when it was put through testing by the ACLU, which announced the results Thursday. The misidentified faces disproportionately belonged to people of color. Responding on its blog, Amazon said the ACLU didn’t test Rekognition with the correct settings, and that its system is meant to help humans make big decisions — not final determinations on its own. Amazon amended the blog post on Friday, Cnet reported, inviting the federal government to recommend rules for how law enforcement uses facial recognition technology.
  • IBM’s Watson gave doctors "unsafe and incorrect" recommendations for cancer treatments, Stat News reported last week, quoting internal IBM documents. The finding blamed both IBM engineers and the doctors who were feeding in training data. IBM told Stat News that Watson Health has since improved.
  • In an earlier case, a self-driving Uber killed a pedestrian in Arizona in March.

The context: For skeptics of deep learning, the leading machine-learning method that powers most commercial AI, these shortcomings belie greater problems ahead.

  • "We shouldn’t mistake pattern recognition for genuine intelligence," Gary Marcus, an NYU professor, tells Axios in an email. "And we shouldn’t be surprised when narrow, shallow intelligence (which is all we have, so far) lets us down."
  • Garrett Kenyon, a scientist at Los Alamos National Laboratory, said in an interview that deep learning can’t grasp abstract concepts, or even reliably count or compare objects.
  • This isn’t the first time an external audit has found bias in deployed face-recognition algorithms. In research published by MIT, researcher Joy Buolamwini tested three companies' face-recognition systems and found that they performed poorly on darker-skinned and female faces. In response, two of the companies — IBM and Microsoft — published improvements to their algorithms.

The other side: Jack Clark, strategy and communications director at OpenAI, said these cases are not marks against deep learning as a technology.

  • "We know DL works," Clark said, using an abbreviation for deep learning.
  • "We also know that DL bugs can be pretty bad and implementing DL systems is hard," he continued.
  • Without more details about the Amazon and IBM incidents, "it's very difficult to make a call as to whether this is due to a flaw in implementation (which would be my assumption) or a flaw in the algorithm itself (which to my mind seems less likely)."

The bottom line: Clark said that AI systems need to be "vigorously and transparently tested in the wild" before they’re put to work in the real world.

Go deeper

Coronavirus dashboard

Illustration: Aïda Amer/Axios

  1. Global: Total confirmed cases as of 7 p.m. ET: 932,605 — Total deaths: 46,809 — Total recoveries: 193,177Map.
  2. U.S.: Total confirmed cases as of 7 p.m. ET: 213,372 — Total deaths: 4,757 — Total recoveries: 8,474Map.
  3. Business updates: Very small businesses are bearing the brunt of the coronavirus job crisis.
  4. World update: Spain’s confirmed cases surpassed 100,000, and the nation saw its biggest daily death toll so far. More than 500 people were reported dead within the last 24 hours in the U.K., per Johns Hopkins.
  5. State updates: Florida and Pennsylvania are the latest states to issue stay-at-home orders — Michigan has more than 9,000 confirmed cases, an increase of 1,200 and 78 new deaths in 24 hours.
  6. Stock market updates: Stocks closed more than 4% lower on Wednesday, continuing a volatile stretch for the stock market amid the coronavirus outbreak.
  7. What should I do? Answers about the virus from Axios expertsWhat to know about social distancingQ&A: Minimizing your coronavirus risk.
  8. Other resources: CDC on how to avoid the virus, what to do if you get it.

Subscribe to Mike Allen's Axios AM to follow our coronavirus coverage each morning from your inbox.

World coronavirus updates: Spain's health care system overloaded

Data: The Center for Systems Science and Engineering at Johns Hopkins, the CDC, and China's Health Ministry. Note: China numbers are for the mainland only and U.S. numbers include repatriated citizens and confirmed plus presumptive cases from the CDC

Two planes with protective equipment arrived to restock Spain’s overloaded public health system on Wednesday as confirmed cases surpassed 100,000 and the nation saw its biggest death toll so far, Reuters reports.

The big picture: COVID-19 cases surged past 900,000 and the global death toll surpassed 45,000 early Wednesday, per Johns Hopkins data. Italy has reported more than 12,000 deaths.

Go deeperArrowUpdated 2 hours ago - Health

FBI sees record number of gun background checks amid coronavirus

Guns on display at a store in Manassas, Va. Photo: Yasin Ozturk / Anadolu Agency via Getty

The FBI processed a record 3.7 million gun background checks in March — more than any month previously reported, according to the agency's latest data.

Driving the news: The spike's timing suggests it may be driven at least in part by the coronavirus.