Creative Commons

Leading artificial intelligence researchers are debating whether machines — once they achieve super-human smarts — should be expected to explain themselves in robust discussion, or if spitting out an exceptional, if mysterious, answer is good enough.

The debate is part of the raging global discussion over a perceived challenge posed by machine intelligence to jobs, the human role in society, geopolitical power — to us.

Battle lines: Peter Norvig, head of research at AI powerhouse Google, has attracted attention with a June 22 speech in Sydney arguing that future intelligent computers shouldn't be expected to explain why they think what they think, since, after all, humans themselves don't do a very good job at explaining the true reasons for their decisions. Others argue that neural networks are simply a different way of thinking, and that even if we should be able to plumb the machine mind, there is no way for us to get at it.

But David Ferrucci, who ran IBM's Watson project for the 2011 Jeopardy challenge and now runs an AI company called Elemental Cognition, asserts that machines have to defend themselves. "I need an explanation. I wanted to be able to critique it," he tells Axios.

Why it matters: Machines that don't explain themselves scare people who worry about godlike computer overlords. But researchers like Ferrucci are posing a more fundamental question: Are we ourselves willing to be held accountable for decisions we base on those of a computer? Unless he knows the computer's rationale, Ferrucci doesn't want to gamble. And others agree: Darpa, the Pentagon's radical research lab, is studying what it calls "Explainable Artificial Intelligence," giving it the acronym XAI.

Let's talk about it: The debate played out on stage at the O'Reilly Artificial Intelligence Conference this week in New York. Ferrucci argued that the best way to get to the kind of AI he is thinking of — the kind you can have a normal exchange of opinions with — is to study and develop a computer that thinks like a human. "[But] there is a level to those dialogues that we are nowhere close to having," he said.

The future: Josh Tennenbaum, a professor at MIT, said the answer is to study young children and develop machine intelligence that learns and thinks like them. He played a video of an 18-month-old child who, confronted with an adult having trouble putting books into a closed cabinet, spontaneously opens its doors and watches to make sure the troubled man succeeds. "That's the heart of intelligence right there. And I think that's the grand challenge for AI," he said. "...If we could reverse engineer what's going on in that kid's mind, just think what we can do with robots and other machines that could really help us out."

The Amazon effect: At this stage commercially, many of us are using Alexa or technology like it at home, but Kris Hammond, chief scientist at Narrative Science, said, "Alexa understands words. It doesn't understand language." Alexa can hold only one idea at a time. If you ask a followup, it does not remember what was just said. "You need to be able to challenge," Hammond said. "Otherwise, we would only be listening."

Go deeper

Updated 8 hours ago - Politics & Policy

Trump's Tucker mind-meld

Photo illustration: Sarah Grillo/Axios. Photo: Roy Rochlin/Getty Images and BRENDAN SMIALOWSKI/AFP via Getty Images

If you want to understand the rhetorical roots of Trump's Independence Day speech at Mount Rushmore, go back and watch Tucker Carlson's monologues for the past six weeks.

Between the lines: Trump — or rather his speechwriter Stephen Miller — framed the president's opposition to the Black Lives Matter protest movement using the same imagery Carlson has been laying out night after night on Fox.

Updated 11 hours ago - Politics & Policy

Coronavirus dashboard

Illustration: Aïda Amer/Axios

  1. Global: Total confirmed cases as of 6 p.m. ET: 11,366,145 — Total deaths: 532,644 — Total recoveries — 6,154,138Map.
  2. U.S.: Total confirmed cases as of 6 p.m. ET: 2,874,396 — Total deaths: 129,870 — Total recoveries: 906,763 — Total tested: 35,512,916Map.
  3. States: Photos of America's pandemic July 4 ICU beds in Arizona hot spot near capacity — Houston mayor warns about hospitals
  4. Public health: U.S. coronavirus infections hit record highs for 3 straight days.
  5. Politics: Former Trump official Tom Bossert says face masks “are not enough”
  6. World: Mexican leaders call for tighter border control as infections rise in U.S.
  7. Sports: Sports return stalked by coronavirus
  8. 1 📽 thing: Drive-in movie theaters are making a comeback.

Bolton's hidden aftershocks

Photo illustration: Sarah Grillo/Axios. Photo: Justin Sullivan/Getty Images

The news media has largely moved on, but foreign government officials remain fixated on John Bolton's memoir, "The Room Where It Happened."

Why it matters: Bolton's detailed inside-the-Oval revelations have raised the blood pressure of allies who were already stressed about President Trump's unreliability.