Doctors want liability shield from AI mistakes
Add Axios as your preferred source to
see more of our stories on Google.
Illustration: Brendan Lynch/Axios. Stock: Getty Images
The rapid deployment of artificial intelligence in hospitals and clinics is leading the nation's biggest physicians group to lobby for new legal protections if AI harms patients.
Why it matters: The use of increasingly sophisticated AI tools and recommendations is blurring the lines around negligence in health care, raising questions about whether doctors should be held responsible for decisions that rely on software.
- There's added urgency as autonomous AI takes on more and more important medical tasks with less direct oversight.
- The question is who's responsible when something goes wrong.
Driving the news: The American Medical Association this week called for federal and state legislation to protect physicians from liability when AI products injure patients, despite physicians using the tools as intended.
- The group also is pressing for new laws mandating that AI companies carry adequate liability insurance and disclose product limitations and performance data.
- Hospitals, health systems, insurers and health tech vendors should be responsible if the required use of specific AI products results in harmful errors, the AMA policy states.
Between the lines: The group's push comes as AI clinical tools are flooding the health system, unlocking new efficiencies but giving algorithms increasing power to make decisions like whether to order a diagnostic test or prescribe an expensive drug.
- Legal and medical experts point to the possibility of triage errors, in which AI systems that process clinical histories, vital signs and imaging can misjudge the severity of a case without the attending doctor's knowledge.
- The AMA also raises the possibility of an AI system making a plausible recommendation for treatment, but with incomplete laboratory results or an outdated diagnoses. In such cases, a physician may be unable to verify the inputs.
- In cases where a patient is harmed, responsibility should fall on AI manufacturers and vendors — and possibly on health systems that require the AI systems' use — instead of through a medical malpractice claim against the physician, the AMA argues.
Context: There already have been lawsuits around algorithm-driven medical decisions, including a wrongful death case in Alabama involving a screening tool for heart abnormalities.
- The software developer wasn't held liable because its licensing agreement left final medical decisions to doctors.
The big picture: Courts around the country are concluding that companies that design, train and deploy AI systems bear responsibility for what the systems do, said University of California, Davis professor Martin Hilbert, who studies the role of digital information and algorithms in society.
- Accountability hinges on factors including who shares responsibility when multiple parties build an AI system, how to prove fault inside an opaque model, and what to do when damages surface only after the system is deployed.
- "Different parties are responsible for different cases, and you can't just hold the end user responsible," he told Axios.
- The AMA wants to address situations when use of an AI system is mandatory.
What we're watching: The Food and Drug Administration is weighing whether a new framework is needed for regulating AI-enabled medical devices.
- But the agency oversees software that meets the definition of a medical device, not all of the AI in health care.
- Hospitals generally support putting the onus on developers and vendors for monitoring the systems' performance, saying the "black box" nature of many systems can make it hard to identify flaws.
- AdvaMed, the trade group representing device makers, is urging the FDA to continue using its current approaches for evaluating generative AI-enabled medical devices. It also wants the agency to avoid post-market monitoring that imposes added burdens regardless of a product's actual risk.
