Illustration: Sarah Grillo/Axios
Siri or Alexa can get you the weather, but don’t expect a conversation. Neither can chatbots (once the next big thing) hold a back-and-forth. But researchers are now developing systems that leapfrog chit-chat to the next frontier: They can argue and play devil's advocate.
Why it matters: Whenever artificial intelligence reaches an advanced stage, far beyond current capabilities, researchers want machines that humans can converse knowledgeably with, and can explain how they reach their conclusions.
What's going on: The field that works on such systems is called "computational argumentation." This summer, a roomful of journalists watched a demonstration in which two human debaters argued with a talking computer, produced by IBM, that used AI-infused computational argumentation to deliver speeches and rebut its opponents’ claims.
Axios spoke with six researchers behind IBM's "Project Debater" to learn how it works, and to track the state of the art in language understanding.
- IBM has gotten a black eye in recent years for — in its critics' view — hyping the promise of its AI in flashy demos, then underdelivering. A particular target has been Watson Health, which IBM sells to hospitals.
- But IBM says the debater is more of a proof-of-concept at this stage, without a current marketing plan.
The context: AI researchers are prone to using games to measure their progress. In 2011, Watson beat a pair of humans at Jeopardy. In 2016, a Google-developed AI system beat the reigning human champion at Go.
- This summer's live debate was a test in the same tradition — but differed in some meaningful ways, said Noam Slonim, who leads the six-year-old IBM debate project.
- A live debate has no clear rules, nor a point system to determine a winner. At best, you can poll a human audience to see which side was most convincing.
- Plus, the art of arguing is built around the subtleties of human speech, a complex beast that computers still have trouble parsing.
"Humans invented language and it's probably the most sophisticated thing we've done so far," said Salim Roukos, an IBM researcher focused on natural language processing.
- To teach the machine, the researchers fed it millions of articles, hundreds of recorded speeches, countless hours of crowdsourced work, and a network of background knowledge.
How it works: The IBM machine picks out factual statements from academic journals and news stories, and arranges them into themes and whether they support or oppose a particular stance.
- Once it’s assigned a debate question, the system assembles a short speech comprised of what — according to the algorithm — are the best arguments and evidence for its side.
- Then, after human opponents make their argument, it identifies the main claims and rebuts.
The entire system is a hodgepodge that combines AI techniques with an extensive collection of knowledge and rules. Some experts say such hybrid systems are the future of AI.
- To make its case, Project Debater can choose from a long list of "principled arguments" that supposedly appeal to human nature. For instance, when the question at hand is about banning something, it can argue that it would spark a black market. If it’s about requiring that people do something, the debater can warn of a resulting backlash.
- The result can seem mechanical and reflexive. To know when to deploy which argument, the system relies on pattern-matching, reaching into its database of recorded debates to guess which argument a human would choose in each situation.
- It should know, for example, that it doesn’t make sense to argue that a ban on public breastfeeding could lead to the creation of a black market, said Ranit Aharonov, the global manager of Project Debater.
IBM’s work on detecting claims and points of view is promising, said Sean Gourley, a former NASA engineer who founded a company focused on natural language.
- These are two difficult problems "right at the edge" of the field, he said.
- But he questioned whether the the work would ever go beyond "performance art pieces."
Reality check: The debater, while handy at marshaling and conveying arguments, is not *intelligent" in a human sense. It does not reason. Like the machines that beat humans at chess, Go, and Jeopardy, it rather muscles its way through mounds of data, without any real thinking at all.
Editor’s note: An earlier version of this article said Project Debater was developed by IBM Watson. It was developed by IBM Research.