Illustration: Aïda Amer/Axios

One of the oddest ways that an AI system can fail is by falling prey to an adversarial attack — a cleverly manipulated input that makes the system behave in an unexpected way.

Why it matters: Autonomous car experts worry that their cameras are susceptible to these tricks: It's been shown that a few plain stickers can make a stop sign look like a "Speed Limit 100" marker to a driverless vehicle. But other high-stakes fields — like medicine — are paying too little attention to this risk.

That's according to a powerhouse of researchers from Harvard and MIT, who published an today article in Science arguing that these attacks could blindside hospitals, pharma companies, and big insurers.

Details: Consider a photo of a mole on a patient's skin. Research has shown that it can be manipulated in a way that's invisible to the human eye, but still changes the result of an AI system's diagnosis from cancerous to non-cancerous.

The big question: Why would anyone want to do this?

  • For Samuel Finlayson, an MD–PhD candidate at Harvard and MIT and the lead author of the new paper, it’s a question of incentives. If someone sending in data for analysis has a different goal than the owner of the system doing the analysis, there's a potential for funny business.
  • We're not talking about a malicious doctor manipulating cancer diagnoses — "There's way more effective ways to kill a person," Finlayson says — but rather an extension of existing dynamics into a near future where AI is involved in billing, diagnosis, and reading medical scans.

Doctors and hospitals already game the insurance billing system — these could be considered proto-adversarial attacks, Finlayson tells Axios.

  • They often bill for more expensive procedures than they performed, in order to make more money, or avoid billing for procedures that they know will land a huge bill in the patient's lap.
  • Insurance companies are already hiring tech firms to put a stop to the practice, often with AI tools. Finlayson sees a future where basic adversarial attacks are used to fool the AI systems into continuing to accept fraudulent claims.
  • Despite this possibility, hospitals and the pharma industry are flying blind, he says. "Adversarial attacks aren't even on the map for them."

But, but, but: These hypotheticals are a bit far-fetched for Matthew Lungren, associate director of the Stanford Center for Artificial Intelligence in Medicine and Imaging.

  • "There are a lot of easier ways to defraud the system, frankly," he tells Axios.
  • But there is an urgent need, Lungren says, to test medical AI systems more rigorously before they're released into the world. Protecting against adversarial attacks is one of the ways experts should shore up algorithms before using them on patients.

Go deeper: Scientists call for rules on evaluating predictive AI in medicine

Go deeper

The CIA's new license to cyberattack

Illustration: Aïda Amer/Axios

In 2018 President Trump granted the Central Intelligence Agency expansive legal authorities to carry out covert actions in cyberspace, providing the agency with powers it has sought since the George W. Bush administration, former U.S. officials directly familiar with the matter told Yahoo News.

Why it matters: The CIA has conducted disruptive covert cyber operations against Iran and Russia since the signing of this presidential finding, said former officials.

3 hours ago - Technology

Tech hits the brakes on office reopenings

Illustration: Annelise Capossela/Axios

Tech was the first industry to send its workers home when COVID-19 first hit the U.S., and it has been among the most cautious in bringing workers back. Even still, many companies are realizing that their reopening plans from as recently as a few weeks ago are now too optimistic.

Why it matters: Crafting reopening plans gave tech firms a chance to bolster their leadership and model the beginnings of a path back to normalcy for other office workers. Their decision to pause those plans is the latest sign that normalcy is likely to remain elusive in the U.S.

The existential threat to small business

Illustration: Eniola Odetunde/Axios

The coronavirus pandemic has changed the game for U.S. businesses, pushing forward years-long shifts in workplaces, technology and buying habits and forcing small businesses to fight just to survive.

Why it matters: These changes are providing an almost insurmountable advantage to big companies, which are positioned to come out of the recession stronger and with greater market share than ever.