Sep 21, 2018

The fuzzy numbers for tracking AV progress

Illustration: Rebecca Zisser/Axios

Autonomous vehicle companies track — and publish, at least in California — a variety of performance metrics, including how many times an operator has to take control of the autonomous system, accidents, and distance traveled autonomously. These convenient figures have become de facto measures of progress, yet the view they offer into the state of AV technology is fuzzy at best.

The bottom line: As AV technology approaches commercialization, companies will have to engage in a dialogue with government and the public — using statistical performance measures as well as qualitative analyses and processes — to demonstrate that autonomous systems are thoughtfully and safely engineered.

The metrics:

  • Disengagements: Absent contextual details, the number of times an operator takes control of the vehicle is a poor indicator of performance. Too few disengagements indicate a lack of instructive situations from which to learn. Too many, especially in similar situations, suggest the system is not learning enough. Rather than minimizing disengagements (because they are publicly reported), teams should monitor them to ensure their technology is being sufficiently challenged and learning the right lessons along the way.
  • Accidents: Egregious accidents can focus needed attention on companies with unsafe cultures or practices. But in aggregate, the lack of reliable baseline numbers for human driving makes it difficult to draw meaningful conclusions for AVs. (The majority of accidents involving human drivers go unreported.) Accident reports might provide limited insight into one aspect of a system’s capabilities, but they don't reflect a system’s overall performance — or the industry’s.
  • Distance: The number of miles driven is probably the most useful metric because real-world performance matters. While simulated miles are hugely valuable, the right road miles present a system with the challenges of unanticipated situations. But all experience is not equal: Less complicated miles, like those on uncongested freeways, are easy to accumulate though less instructive per mile than those in a challenging urban environment.

Chris Urmson is a co-founder and the CEO of Aurora.

Read more stories like this in our new weekly Axios Autonomous Vehicles newsletter. Sign up here.

Go deeper

Trump administration asks Congress for $2.5 billion to fight coronavirus

President Trump with Health and Human Services Secretary Alex Azar at the White House in September. Photo: Chip Somodevilla/Getty Images

The Trump administration sent a letter to Congress Monday requesting a funding commitment of at least $2.5 billion to help combat the spread of the novel coronavirus.

Details: The request for a lump sum account for the Department of Health and Human Services includes $1.25 billion in new funds to fight COVID-19 and $535 would come from untouched funds for the Ebola virus.

Go deeperArrow2 hours ago - Health

WHO won't call coronavirus a pandemic as cases spread

Data: The Center for Systems Science and Engineering at Johns Hopkins, the CDC, and China's Health Ministry. Note: China numbers are for the mainland only and U.S. numbers include repatriated citizens.

The World Health Organization will not yet call the coronavirus a pandemic, claiming that needs across affected countries are too varied and the classification would increase fear, per a briefing Monday.

The big picture: As South Korea and Italy stepped up emergency measures in efforts to thwart the spread of the virus, WHO expressed concern about infections with no clear link to China. COVID-19 has killed at least 2,620 people and infected almost 80,000 others, with all but 27 deaths occurring in mainland China.

Go deeperArrowUpdated 4 hours ago - Health

The global scramble to contain the coronavirus

Taking precaution, in the Philippines. Photo: Ezra Acayan/Getty Images

The coronavirus is spreading quickly in cities nowhere near Wuhan, China, and the window to prevent a global pandemic is narrowing.

Zoom in: Here's a look at what comes with a coronavirus outbreak in communities outside China that have been hardest hit so far.

Go deeperArrow4 hours ago - World