Exclusive: FDA considers assessing AI devices like doctors
Add Axios as your preferred source to
see more of our stories on Google.
Illustration: Gabriella Turrisi/Axios
The Food and Drug Administration is considering evaluating generative AI-enabled medical devices through a "competency-based approach," similar to how doctors are evaluated, the agency writes in a discussion paper first shared with Axios.
Why it matters: Emerging medical technologies are challenging the existing framework for overseeing medical devices, prompting calls for new approaches.
Driving the news: The FDA's discussion paper raises considerations for both preapproval and postmarket regulation of AI-enabled devices and solicits stakeholder feedback.
- The document isn't intended to serve as guidance for device makers and notes that the agency is still in the preliminary phase of its effort.
- "We are launching a transparent process to inform the development of an approach that safeguards patients and consumers, advances innovation and serves as a potential model for regulators around the world," Michelle Tarver, director of the FDA Center for Devices and Radiological Health, said in a statement.
- The agency's "ultimate goal is to enable a nimble regulatory approach that employs least burdensome principles and allows patients timely access to safe and effective medical devices," the paper states.
The big picture: There's been an explosion in AI-enabled health care tools directly involved in clinical care, and the regulatory structure so far hasn't kept up.
- New technologies range from information tools to devices designed to provide patient care, either with or without a doctor in the loop.
- "Artificial intelligence is transforming medicine, and the United States must lead in shaping how this technology is developed and used safely and responsibly," acting FDA commissioner Kyle Diamantas said in a statement.
Between the lines: Regulatory experts — and now the FDA — note that generative AI devices are substantially different from other regulated medical devices, particularly because of the variation in their outputs and because they evolve over time.
- That requires a shift in how the agency evaluates such devices before they're on the market and also creates a need for rigorous postmarket monitoring.
What's inside: The FDA's discussion paper begins by outlining a method for evaluating the risk of a GenAI device, measuring the type of activity performed by a device against the severity of the consequences from incorrect outputs.
- It then discusses considerations for premarket evaluation, including a competency-based approach that is "inspired, at a high level, by how human clinicians are evaluated and credentialed."
- That approach would consist of setting benchmarks and confirming performance in clinical settings.
- The discussion paper weighs approaches to tracking how the devices perform once they're on the market, and states the agency is considering "whether it is appropriate to accept greater premarket uncertainty regarding a GenAI-enabled device's benefit-risk profile through greater reliance on postmarket monitoring."
The intrigue: One big question the paper poses is what standard GenAI devices should be compared against.
- The agency says that for many devices, "performance might be compared to that of a panel of qualified clinicians whose consensus reflects the applicable standard of care, or to that of a median clinician in practice."
