Oct 4, 2018

IBM’s argumentative AI

Illustration: Sarah Grillo/Axios

Siri or Alexa can get you the weather, but don’t expect a conversation. Neither can chatbots (once the next big thing) hold a back-and-forth. But researchers are now developing systems that leapfrog chit-chat to the next frontier: They can argue and play devil's advocate.

Why it matters: Whenever artificial intelligence reaches an advanced stage, far beyond current capabilities, researchers want machines that humans can converse knowledgeably with, and can explain how they reach their conclusions.

What's going on: The field that works on such systems is called "computational argumentation." This summer, a roomful of journalists watched a demonstration in which two human debaters argued with a talking computer, produced by IBM, that used AI-infused computational argumentation to deliver speeches and rebut its opponents’ claims.

Axios spoke with six researchers behind IBM's "Project Debater" to learn how it works, and to track the state of the art in language understanding.

  • IBM has gotten a black eye in recent years for — in its critics' view — hyping the promise of its AI in flashy demos, then underdelivering. A particular target has been Watson Health, which IBM sells to hospitals.
  • But IBM says the debater is more of a proof-of-concept at this stage, without a current marketing plan.
IBM's talking machine, and a human debater. Photo: IBM

The context: AI researchers are prone to using games to measure their progress. In 2011, Watson beat a pair of humans at Jeopardy. In 2016, a Google-developed AI system beat the reigning human champion at Go.

  • This summer's live debate was a test in the same tradition — but differed in some meaningful ways, said Noam Slonim, who leads the six-year-old IBM debate project.
  • A live debate has no clear rules, nor a point system to determine a winner. At best, you can poll a human audience to see which side was most convincing.
  • Plus, the art of arguing is built around the subtleties of human speech, a complex beast that computers still have trouble parsing.

"Humans invented language and it's probably the most sophisticated thing we've done so far," said Salim Roukos, an IBM researcher focused on natural language processing.

  • To teach the machine, the researchers fed it millions of articles, hundreds of recorded speeches, countless hours of crowdsourced work, and a network of background knowledge.

How it works: The IBM machine picks out factual statements from academic journals and news stories, and arranges them into themes and whether they support or oppose a particular stance.

  • Once it’s assigned a debate question, the system assembles a short speech comprised of what — according to the algorithm — are the best arguments and evidence for its side.
  • Then, after human opponents make their argument, it identifies the main claims and rebuts.

The entire system is a hodgepodge that combines AI techniques with an extensive collection of knowledge and rules. Some experts say such hybrid systems are the future of AI.

  • To make its case, Project Debater can choose from a long list of "principled arguments" that supposedly appeal to human nature. For instance, when the question at hand is about banning something, it can argue that it would spark a black market. If it’s about requiring that people do something, the debater can warn of a resulting backlash.
  • The result can seem mechanical and reflexive. To know when to deploy which argument, the system relies on pattern-matching, reaching into its database of recorded debates to guess which argument a human would choose in each situation.
  • It should know, for example, that it doesn’t make sense to argue that a ban on public breastfeeding could lead to the creation of a black market, said Ranit Aharonov, the global manager of Project Debater.

IBM’s work on detecting claims and points of view is promising, said Sean Gourley, a former NASA engineer who founded a company focused on natural language.

  • These are two difficult problems "right at the edge" of the field, he said.
  • But he questioned whether the the work would ever go beyond "performance art pieces."

Reality check: The debater, while handy at marshaling and conveying arguments, is not *intelligent" in a human sense. It does not reason. Like the machines that beat humans at chess, Go, and Jeopardy, it rather muscles its way through mounds of data, without any real thinking at all.

Editor’s note: An earlier version of this article said Project Debater was developed by IBM Watson. It was developed by IBM Research.

Go deeper

Coronavirus dashboard

Illustration: Sarah Grillo/Axios

  1. Global: Total confirmed cases as of 1 p.m. ET: 1,237,420 — Total deaths: 67,260 — Total recoveries: 252,944Map.
  2. U.S.: Total confirmed cases as of 1 p.m. ET: 312,762 — Total deaths: 9.132 — Total recoveries: 15,044Map.
  3. Public health latest: CDC launches national trackers and recommends face coverings in public. Federal government will cover costs of COVID-19 treatment for uninsured. Surgeon general says this week will be "our Pearl Harbor, our 9/11 moment."
  4. 2020 latest: "We have no contingency plan," Trump said on the 2020 Republican National Convention. Biden says DNC may have to hold virtual convention.
  5. States updates: New York Gov. Andrew Cuomo said the state "literally going day-to-day" with supplies.
  6. Work update: Employees still going to work face temperature checks, distanced work stations, protective devices and mass absences.
  7. What should I do? Pets, moving and personal health. Answers about the virus from Axios expertsWhat to know about social distancingQ&A: Minimizing your coronavirus risk.
  8. Other resources: CDC on how to avoid the virus, what to do if you get it.

Subscribe to Mike Allen's Axios AM to follow our coronavirus coverage each morning from your inbox.

Cuomo says New York is "literally going day-to-day with our supplies"

New York Gov. Andrew Cuomo said in a press conference on Sunday that New York is struggling to maintain medical supplies while combatting the novel coronavirus — operating "literally" on a "day-to-day" basis.

Why it matters: New York City has become an epicenter of the coronavirus outbreak, facing mass quarantines and stay-at-home orders. Cuomo said Saturday that New York reported 630 new deaths in 24 hours — an "all-time increase" that beat the previous day's record of 562 deaths.

Illinois governor: "The president does not understand the word 'federal'"

Illinois Gov. J.B. Pritzker said on CNN's "State of the Union" Sunday that President Trump's comments about the federal government's stockpile of medical equipment suggest he "does not understand the word 'federal.'"

Why it matters: White House adviser Jared Kushner argued at a press briefing last week that the "notion of the federal stockpile was it’s supposed to be our stockpile; it’s not supposed to be state stockpiles that they then use."