Watch out for self-driving cars: They're watching you

Autonomous vehicles don't just use cameras to help steer themselves. To keep improving, they're also capturing and storing images of everything that surrounds them — which means they might catch you on camera if you're in the vicinity.

Why it matters: This is a big issue that privacy experts are just starting to think about. It's not clear who else might see those images — and without concrete rules on how data collected outside the vehicle may be used, bystanders' privacy could be at risk.

The big picture: Connected vehicles pose all sorts of potential privacy issues for car owners:

• Your car's navigation system needs to know where you are to give directions.
• To enable hands-free dialing, you usually sync your phone's contacts to the car.
• When you use Apple CarPlay or Google's Android Auto, you may be exposing data from your car to third-party app providers.

In 2014, 20 automakers signed a voluntary set of automotive privacy principles, effective with 2017 models, agreeing to ask permission before using or sharing sensitive information about occupants, and to limit what they share with government and law enforcement.

None of those principles governs data collected about bystanders outside a vehicle. If the cameras on a fleet of AVs plying your neighborhood routinely capture your daily movements, you could be sharing a lot more than you realize.

"If someone knows where you’ve been — an abortion clinic, a mosque or church, or leaving a lover's apartment — it reveals private information about you."

What we're hearing: We asked a half dozen automakers and AV tech companies how long they keep camera footage, how it is stored and who has access. The handful that replied assured us they take privacy very seriously, but didn't offer any specifics about bystander privacy.

People can't expect privacy in public places, but there are some related precedents for how to limit potential exposure:

• Some states ban video surveillance by closed-circuit TV cameras and all states require posted warnings when security cameras are present.

• Google's StreetView app generated controversy until the company started blurring faces and license plates captured incidentally.
• Drones raise privacy concerns but the industry says there are ways to limit the collection and retention of personal information.

Yes, but: The situation may be different for AVs that rely on huge stored databases to train the car's algorithms, Smith notes.

• Extensive, repeated mapping of a neighborhood is needed to develop the technology.
• To design safer systems, it might be important to clearly see a pedestrian's face to know whether they are paying attention to traffic.

The bottom line: It's possible to retain only safety-critical data while discarding other sensitive information, privacy experts say, but steps must be started now.

• It would require asking questions about what camera data is collected, how it's used, with whom it's shared, how much is retained, and how it's stored and protected.
• Then, AV developers can consider the tradeoffs between safety and privacy, and what steps can be taken to mitigate risk.