Pedestrians aren't always paying attention. Photo: Ford

Researchers at the University of Michigan are studying human body language to teach self-driving cars to recognize and predict pedestrian movements with greater precision than current technologies.

Why it matters: People don't always pay attention when crossing the street, so AVs need to be on the lookout for distracted pedestrians, not just other cars on the road.

"If a pedestrian is playing with their phone, you know they're distracted. Their pose and where they're looking is telling you a lot about their level of attentiveness. It's also telling you a lot about what they're capable of doing next."
— Ram Vasudevan, assistant professor of mechanical engineering, Michigan

How it works: Using data collected by vehicles through cameras, lidar and GPS, the researchers captured video snippets of humans in motion and then recreated them in 3D computer simulation.

  • This enabled them to create a "biomechanically inspired recurrent neural network" that catalogs human movements.
  • By focusing on humans' gait, body symmetry and foot placement, they can predict what pedestrians might do next and train self-driving cars to recognize behavior.

Background: Until now, most machine learning for AVs has relied on still images.

  • If you show a computer enough photos of a stop sign it will eventually come to recognize stop signs in the real world.

What's next: By using video clips that run for several seconds, Michigan's system can study the first half of the snippet to make its predictions, and then verify the accuracy with the second half.

  • The researchers said they could predict a pedestrian's location within 10 centimeters after one second and less than 80 centimeters after 6 seconds. All other comparison methods were up to 7 meters off.
  • "We're [now] better at figuring out where a person is going to be," says Matthew Johnson-Roberson, associate professor in Michigan's naval architecture and marine engineering department.

Go deeper

Updated 3 hours ago - Politics & Policy

Coronavirus dashboard

Illustration: Aïda Amer/Axios

  1. Global: Total confirmed cases as of 4:30 p.m. ET: 21,020,216 — Total deaths: 761,393— Total recoveries: 13,048,303Map.
  2. U.S.: Total confirmed cases as of 4:30 p.m. ET: 5,289,323 — Total deaths: 167,948 — Total recoveries: 1,774,648 — Total tests: 64,831,306Map.
  3. Health: CDC: Survivors of COVID-19 have up to three months of immunity Fauci believes normalcy will return by "the end of 2021" with vaccine — The pandemic's toll on mental health.
  4. Business: How small businesses got stiffed — Unemployment starts moving in the right direction.
  5. Cities: Coronavirus pandemic dims NYC's annual 9/11 Tribute in Light.
  6. Politics: Biden signals fall strategy with new ads.

Harris: "Women are going to be a priority" in Biden administration

Sen. Kamala Harris at an event in Wilmington, Del. Photo: Drew Angerer/Getty Images

In her first sit-down interview since being named Joe Biden's running mate, Sen. Kamala Harris talked about what she'll do to fight for women if elected VP, and how the Democrats are thinking about voter turnout strategies ahead of November.

What they're saying: "In a Biden-Harris administration women are going to be a priority, understanding that women have many priorities and all of them must be acknowledged," Harris told The 19th*'s Errin Haines-Whack.

Facebook goes after Apple

Illustration: Lazaro Gamio/Axios

Facebook is seeking to force a face-off with Apple over its 30% in-app purchase commission fee, which Facebook suggests hurts small businesses struggling to get by during the pandemic.

The big picture: Facebook has never publicly gone after Apple, a key strategic partner, this aggressively. Both companies face antitrust scrutiny, which in Apple's case has centered on the very fee structure Facebook is now attacking.