Feb 13, 2019

Another challenge for self-driving cars: Learning human body language

Pedestrians aren't always paying attention. Photo: Ford

Researchers at the University of Michigan are studying human body language to teach self-driving cars to recognize and predict pedestrian movements with greater precision than current technologies.

Why it matters: People don't always pay attention when crossing the street, so AVs need to be on the lookout for distracted pedestrians, not just other cars on the road.

"If a pedestrian is playing with their phone, you know they're distracted. Their pose and where they're looking is telling you a lot about their level of attentiveness. It's also telling you a lot about what they're capable of doing next."
— Ram Vasudevan, assistant professor of mechanical engineering, Michigan

How it works: Using data collected by vehicles through cameras, lidar and GPS, the researchers captured video snippets of humans in motion and then recreated them in 3D computer simulation.

  • This enabled them to create a "biomechanically inspired recurrent neural network" that catalogs human movements.
  • By focusing on humans' gait, body symmetry and foot placement, they can predict what pedestrians might do next and train self-driving cars to recognize behavior.

Background: Until now, most machine learning for AVs has relied on still images.

  • If you show a computer enough photos of a stop sign it will eventually come to recognize stop signs in the real world.

What's next: By using video clips that run for several seconds, Michigan's system can study the first half of the snippet to make its predictions, and then verify the accuracy with the second half.

  • The researchers said they could predict a pedestrian's location within 10 centimeters after one second and less than 80 centimeters after 6 seconds. All other comparison methods were up to 7 meters off.
  • "We're [now] better at figuring out where a person is going to be," says Matthew Johnson-Roberson, associate professor in Michigan's naval architecture and marine engineering department.

Go deeper

Biden formally secures Democratic presidential nomination

Joe Biden speaks at Delaware State University's student cente on June 5. Photo: Jim Watson/AFP via Getty Images

Former Vice President Joe Biden became the formal Democratic presidential nominee on Friday evening, per AP.

The big picture: Biden has been the presumptive frontrunner to take on President Trump since Sen. Bernie Sanders suspended his campaign in early April.

Updated 5 hours ago - Politics & Policy

Coronavirus dashboard

Illustration: Sarah Grillo/Axios

  1. Global: Total confirmed cases as of 9 p.m. ET: 6,724.516 — Total deaths: 394,018 — Total recoveries — 2,996,832Map.
  2. U.S.: Total confirmed cases as of 9 p.m. ET: 1,894,753 — Total deaths: 109,042 — Total recoveries: 491,706 — Total tested: 19,231,444Map.
  3. Public health: WHCA president says White House violated social-distancing guidelines to make reporters "a prop" — Jailing practices contribute to spread.
  4. Sports: How coronavirus could reshuffle the sports calendar.
  5. Jobs: Better-than-expected jobs report boosts stock market.
  6. Media: The Athletic lays off 8% of staff, implements company-wide pay cut.

Scoop: German foreign minister to travel to Israel with warning on annexation

Heiko Maas. Photo: Michael Kappeler/picture alliance via Getty Images

German Foreign Minister Heiko Maas is expected to travel to Israel next week to warn that there will be consequences if Israeli leaders move forward with plans to annex parts of the West Bank, Israeli officials and European diplomats tell me.

Why it matters: Israeli and European officials agree that if Israel goes ahead with unilateral annexation, the EU will respond with sanctions.