Illustration: Rebecca Zisser/Axios

Simulation has been critical to speeding up AV development, especially for motion planning and control algorithms. But in these simulations, vehicles are either not perceiving the simulated world at all or perceiving only scenarios that have been previously encountered on the road.

Why it matters: Using these approaches, vast amounts of on-road data will need to be gathered in order to reach the edges of a perception module's capabilities. An additional layer of simulation for AV sensors and perception systems could use synthetic data to accelerate the development of AVs.

Background: AV simulations test different modules of the self-driving software using map data and information about obstacles on the road.

  • In re-simulations, saved data from on-road drives can be fed back through the software. AV engineers can develop test scenarios to benchmark software performance and then compare the results after making updates.

What's needed: Simulating sensor models as well could help in several areas.

  • Optimizing sensor placement on vehicles. Being able to position and visualize the output from sensors is easier and faster using software than real hardware.
  • Creating data sets to train for rare situations. Simulations can model any imaginable scenario, however unlikely.
  • Feeding perception algorithms with synthetic sensor data. This allows engineers to continuously create new scenarios and test their algorithms, rather than waiting for new on-road data.

The catch: Sensor simulation for perception algorithms is difficult because each sensor type (lidar, radar, camera, infrared) has different levels of complexity.

  • In an ideal case, a real hardware sensor’s information could be reproduced using purely software. This requires modeling the underlying physics of each sensor and carefully constructing the 3D environment in which the synthetic sensors “operate.”
  • Synthetic data could then be passed to perception algorithms in a form that they can’t differentiate from real data.

These simulation platforms have to be capable of creating and rendering large photorealistic environments in real time.

  • Sensor tasks add to the load: Properly simulating radar or lidar can require casting millions of synthetic rays per second.

The bottom line: Sensor simulation for perception is still a new process, but could fill an important gap in the development cycle of AV software.

Chris Gundling is a senior software engineer at Applied Intuition, an AV simulation software company.

Go deeper

Updated 1 hour ago - Health

8 states set single-day coronavirus case records last week

Data: Compiled by Axios; Map: Danielle Alberti/Axios

8 states set new highs last week for coronavirus infections recorded in a single day, according to the COVID Tracking Project (CTP) and state health departments. Montana, West Virginia, and Wyoming surpassed records from the previous week.

Why it matters: Cases and hospitalizations are rising in Michigan, a state that initially fought the pandemic with strict mitigation efforts, alongside states that took less action against the spread of the virus this spring.

Updated 2 hours ago - Politics & Policy

Coronavirus dashboard

Illustration: Annelise Capossela/Axios

  1. Politics: Trump says if Biden's elected, "he'll listen to the scientists"Herd immunity claims by top Trump adviser are "pseudoscience," infectious-disease expert says.
  2. Map: 38 states, D.C. see surge in cases.
  3. Health: Coronavirus hospitalizations are on the rise — Fauci says he's "absolutely not" surprised Trump got coronavirus.
  4. Business: Consumer confidence surveys show Americans are getting nervousHow China's economy bounced back from coronavirus.
  5. Sports: We've entered the era of limited fan attendance.
  6. Education: Why education technology can’t save remote learning.
Dion Rabouin, author of Markets
4 hours ago - Economy & Business

The Fed is starting to question its own policies

Illustration: Aïda Amer/Axios

Several officials at the Fed are beginning to worry about asset bubbles and excessive risk-taking as a result of their extraordinary policy interventions, James Politi writes for the Financial Times, citing interviews with multiple Fed presidents and members of the Board of Governors.

Details: Some are now pushing for "tougher financial regulation" as concerns grow that monetary policy is "encouraging behavior detrimental to economic recovery and creating pressure for additional bailouts."