Feb 13, 2019

Another challenge for self-driving cars: Learning human body language

Pedestrians aren't always paying attention. Photo: Ford

Researchers at the University of Michigan are studying human body language to teach self-driving cars to recognize and predict pedestrian movements with greater precision than current technologies.

Why it matters: People don't always pay attention when crossing the street, so AVs need to be on the lookout for distracted pedestrians, not just other cars on the road.

"If a pedestrian is playing with their phone, you know they're distracted. Their pose and where they're looking is telling you a lot about their level of attentiveness. It's also telling you a lot about what they're capable of doing next."
— Ram Vasudevan, assistant professor of mechanical engineering, Michigan

How it works: Using data collected by vehicles through cameras, lidar and GPS, the researchers captured video snippets of humans in motion and then recreated them in 3D computer simulation.

  • This enabled them to create a "biomechanically inspired recurrent neural network" that catalogs human movements.
  • By focusing on humans' gait, body symmetry and foot placement, they can predict what pedestrians might do next and train self-driving cars to recognize behavior.

Background: Until now, most machine learning for AVs has relied on still images.

  • If you show a computer enough photos of a stop sign it will eventually come to recognize stop signs in the real world.

What's next: By using video clips that run for several seconds, Michigan's system can study the first half of the snippet to make its predictions, and then verify the accuracy with the second half.

  • The researchers said they could predict a pedestrian's location within 10 centimeters after one second and less than 80 centimeters after 6 seconds. All other comparison methods were up to 7 meters off.
  • "We're [now] better at figuring out where a person is going to be," says Matthew Johnson-Roberson, associate professor in Michigan's naval architecture and marine engineering department.

Go deeper

#MeToo gets Weinstein

A man carries out Weinstein's walker. Photo: Johannes Eisele/AFP/Getty Images

Hollywood titan Harvey Weinstein is now a convicted rapist, two years and four months after accusations against him helped ignite the #MeToo movement.

Why it matters: To date, #MeToo has resulted in hundreds of powerful men losing their jobs. Seven have been criminally convicted, with four others still facing charges.

JPMorgan Chase to pull support for some fossil fuels

Illustration: Sarah Grillo/Axios

JPMorgan Chase said Monday that it won’t directly finance new oil and gas development in the Arctic and will significantly curtail its financing of the extraction and burning of coal.

Why it matters: JPMorgan is the world’s largest funder of fossil-fuel companies, according to a report by the Rainforest Action Network (RAN). The announcement follows similar moves by other big banks and investment firms, including Goldman Sachs and BlackRock.