Mar 15, 2018

Yejin Choi: Trying to give AI some common sense

Photo illustration: Axios Visuals

Artificial intelligence researchers have tried unsuccessfully for decades to give machines the common sense needed to converse with humans and seamlessly navigate our always-changing world. Last month, Paul Allen announced he is investing another $125 million into his Allen Institute for Artificial Intelligence (AI2) in a renewed effort to solve one of the field's grand challenges.

Axios spoke with Yejin Choi, an AI researcher from the University of Washington and AI2 who studies how machines process and generate language. She talked about how they're defining common sense, their approach to the problem and how it's connected to bias.

How do you define common sense?

"Common sense is fairly trivial everyday knowledge that we have about people and about the world. It's knowledge about how the world works — how people think, what motivates them, how they act, and why they do what they do."

"Imagine there's a robot in your household in the future, and you want to store leftover pie in a container. The robot should pick a container that's large enough to store that pie, and today that spatial reasoning relative to physical properties of different objects in the world and how you interact with them are not quite well represented in these system models."

Why common sense poses a challenge to machines:

"We have a world model in our mind when we do daily operations. AI systems today, despite tremendous advancement in recent years, they are not very good at generalizing out of pure example, so they tend to be very, very task specific, and very domain specific."

"A machine translation system may seem like it understands some language enough to translate into another language, but actually there's not that much understanding happening, per se, because that syntax knowledge cannot be reused for making very trivial small talk with a human, for example."

Their approach:

"We have this commonsense knowledge without our parents or teachers having to enumerate all of it one by one. Nobody told us that elephants are usually bigger than butterflies, however we can reason about it. You ask me that question, I can think about it, and I can answer that question even though I've never seen that statement explicitly written anywhere."

"We're taking a similar approach. It may be possible that we can learn to answer these sort of questions, even including those that we've never seen before. That's fundamentally the ability that AI systems needs to have — dealing with unknowns and previously unseen situations."

What data is needed to make common sense models?

"There is a paper called Verb Physics, and in that work the dataset is basically a combination of a lot of natural language documents — a huge corpus of how people use language and from that we look for patterns. For example, what kind of things do I throw? What kind of things do I enter into? I enter my house. I exit my house. And, that sort of implies that my house must be bigger than me for me to enter into and exit from."

"So, we can infer different action dynamics, preconditions, and post conditions — all those different physical objects, for me to do some action involving them."

"The short term goal is to develop a common sense benchmark dataset. Then, the ultimate goal is to acquire knowledge that's good enough to do well in that benchmark dataset. That's step one."

The issue of AI and human bias:

"We showed [in a study of movie scripts last year] how women in movies carry much less power compared to men. It's the kind of actions that they do and the kind of language they use when they speak. Men usually fight and they do stuff, they save the world. Women, on the other hand, they tend to wait, they are being watched, and they look pretty. What they do tends to be pretty passive."

"It's one of my passions to develop AI technology that can detect all these biases in humans and also, ideally, be able to correct them in the future."

How is bias connected to common sense?

"These are connected in that the way bias is coming across, often times can be inclusive or implied. Current models are much better at understanding what's explicitly stated, but less good at anticipating what's not said. It's good to be catching some of the explicit biases, but it's important to also detect all of the implied ones because that still influences us. The ability to read between the lines ultimately is what requires common sense, so that's the connection between the two."

Go deeper

Coronavirus dashboard

Illustration: Sarah Grillo/Axios

  1. Global: Total confirmed cases as of 8 p.m. ET: 5,682,389 — Total deaths: 354,944 — Total recoveries — 2,337,385Map.
  2. U.S.: Total confirmed cases as of 8 p.m. ET: 1,697,459 — Total deaths: 100,276 — Total recoveries: 391,508 — Total tested: 15,192,481Map.
  3. Public health: Fauci says data is "really quite evident" against hydroxychloroquine — Nearly half of Americans say someone in their household has delayed medical care.
  4. States: California hospitals strained by patients in MexicoTexas Supreme Court blocks mail-in expansion to state voters.
  5. Business: MGM plans to reopen major Las Vegas resorts in June — African American business owners have seen less relief from PPP, Goldman Sachs says.
  6. Tech: AI will help in the pandemic — but it might not be in time for this one.
  7. 1 🎶 thing: Local music venues get rocked by coronavirus.
  8. 🎧 Podcast: Trump vs. Twitter ... vs. Trump.
  9. What should I do? When you can be around others after contracting the coronavirus — Traveling, asthma, dishes, disinfectants and being contagiousMasks, lending books and self-isolatingExercise, laundry, what counts as soap — Pets, moving and personal healthAnswers about the virus from Axios expertsWhat to know about social distancingHow to minimize your risk.
  10. Other resources: CDC on how to avoid the virus, what to do if you get it, the right mask to wear.

Subscribe to Mike Allen's Axios AM to follow our coronavirus coverage each morning from your inbox.

Updated 31 mins ago - Politics & Policy

Top Senate Democrat says State Dept. is working on new Saudi arms deal

Secretary of State Mike Pompeo briefs reporters on May 20. Photo: Nicholas Kamm/pool/AFP via Getty Images

Senate Foreign Relations ranking member Bob Menendez (D-N.J.) wrote in a CNN op-ed on Wednesday that he learned that the State Department is currently working to sell thousands of additional precision-guided bombs to Saudi Arabia.

Why it matters: Democrats say that Steve Linick, the State Department inspector general who was ousted on Secretary of State Mike Pompeo's recommendation, was investigating the administration's previous effort to sell weapons to Saudi Arabia without congressional approval.

U.S. coronavirus death toll crosses 100,000

Data: Johns Hopkins University; Chart: Danielle Alberti/Axios

More than 100,000 Americans have died of the coronavirus, according to data from Johns Hopkins — a terrible milestone that puts the death toll far beyond some of the most tragic events in U.S. history.

By the numbers: The death toll from COVID-19 now stands at more than 34 times the number of people who died on 9/11.