May 3, 2024 - Technology

VR is powering the road trip game of the future

Animated GIF of an extended-reality video game showing a race car dodging cars in the real world.

In the Valeo Racer video game, the real world is your race track. Image courtesy of Valeo

Carmakers could soon offer vehicle entertainment systems that seamlessly blend the real world with the digital world.

Why it matters: As cars become more automated and connected, passengers will be looking to enhance their entertainment experience with technologies like mixed reality — whether it's to play high-tech video games or to invite virtual passengers on a real road trip.

Driving the news: I recently got to try out a new extended-reality racing video game developed by Valeo, a French automotive supplier that makes hardware and software for assisted-driving systems.

  • Valeo Racer turned our demo car, a Volvo XC90, into a real-time participant in the video game, using the world around it as a digital racetrack.
  • From the back seat, I used a video game controller to navigate an animated race car in and out of real traffic, while trying to collect coins and avoid obstacles like traffic cones or the occasional stray tire "falling off" a real car.

At times, I feared my little race car would be squished by the giant SUVs and pickups around me — but since it was only a game, it didn't really matter.

  • I could drive as recklessly as I wanted, jumping over obstacles or spinning out, without any real-world consequences.
Photo of the author and a Valeo executive playing an XR video game in a moving car.
I'm virtually racing against Geoffrey Bouquot, CTO at Valeo, on real streets in Austin, Texas. Photo courtesy of Valeo

Between the lines: The game is a collaboration between Valeo and gaming software developer Unity, which generated the 3D graphics and other game elements.

  • Because the game integrates digital elements into a live feed of the vehicle's real surroundings — including buildings, roads and traffic — the gaming experience and challenges are different every time.
  • While some virtual and mixed reality headsets can create motion sickness in a moving car, there is no such sensation with Valeo Racer because the game is synchronized with the car's movement.
  • Players can even race against passengers in other real cars on the road.

The big picture: Such experiences are possible because of the emergence of what's being called the "software-defined vehicle."

  • Increasingly automated cars are packed with software and sensors to understand their environment, including cameras, lidar, radar and ultrasound.
  • Data collected from those sensors is aggregated in two or three high-powered central processing units, versus hundreds of individual computers as in the past.
  • That makes it easier to repurpose those sensors and computing power to support new entertainment experiences, like augmented reality.

What they're saying: Valeo is not going to shift to making video games for cars, says Christopher Nowakowski, Valeo's research and innovation manager.

  • "We want to build the systems that underlie this experience so that other people can build on top of it."

What's next: Valeo also envisions enhanced video-conferencing with augmented reality — think FaceTime for the car.

  • Using both internal and external cameras, your friend can see you — and where you're driving.
  • It's not safe, of course, to conduct a video call while driving. But it could make sense to share your experience when you're a passenger in a future self-driving car.
  • Or imagine bringing Grandma along virtually on that family vacation.

The bottom line: A whole new passenger experience is coming to self-driving cars.

Go deeper