Jun 30, 2021 - Technology

Thermal cameras are the new eyes for automated driving features

Side by side images showing how FLIR's thermal imaging technology can improve visibility for assisted-driving systems.
FLIR's thermal imaging sensors can detect objects ordinary cameras might not see. Photo: courtesy of Teledyne FLIR

New imaging technology is designed to address a dangerous blind spot when it comes to today's assisted driving systems: they don't always brake for pedestrians.

Why it matters: About 6,200 pedestrians are killed by motor vehicles every year, according to federal statistics, accounting for 16% of all traffic deaths. Three-fourths of those deaths occur at night.

  • Millions of cars are equipped with automatic emergency braking systems, which are supposed to detect and avoid pedestrians. But they are often "ineffective" at night, AAA testing found.
  • These systems are powered by radar and visible cameras, which have trouble seeing in low-light, glaring sun or bad weather.
  • Adding thermal sensors could greatly improve pedestrian safety in these situations, as recent tests performed by VSI Labs, a technical advisor to self-driving car developers, found.

What's happening: Oregon-based Teledyne FLIR Systems, a maker of thermal-imaging and night-vision technology for the military and others, has developed a new type of sensor that can simultaneously detect both light and heat.

  • In a recent demo, I saw how well the so-called "thermal camera" picked up vulnerable road users like a woman crossing the street carrying a child's slide or smokers taking a break in the shadows of a parking garage.
  • The "blended thermal/visible technology" can also help detect humans and animals for forthcoming augmented reality systems in cars, FLIR's John Eggert told me.

How it works: Regular cameras make pictures from visible light but can struggle in high-contrast light/dark environments.

  • Thermal cameras make pictures from heat energy, not visible light, so they're largely unaffected by sun glare and shadows.
  • Heat (also called infrared, or thermal, energy) and light are both parts of the electromagnetic spectrum, but a camera that can detect visible light won’t see thermal energy, and vice versa.

The bottom line: Sensor fusion — combining multiple perception systems into one — will help improve how cars of the future will see.

Go deeper