Technological advances — and a controversial policy experiment in Utah — are already challenging the notion that AI won't replace doctors.

Why it matters: There are a growing number of voices sounding the "not ready" alarm, warning that unsupervised AI-provided medical care is unchecked and poses serious legal concerns.

• At the same time, the once far-fetched idea that AI will take over some of the clinical work currently performed by human providers will soon be reality.

Driving the news: This isn't a debate for the future. It's playing out right now in Utah.

• The state has partnered with health startup Doctronic in a pilot program that will allow AI systems to automate some prescription refills for chronic conditions, with some human oversight.
• Though the initiative is still in the first phase, which requires human review and approval of every AI-renewed prescription, the state medical board last month called for the program to be suspended and said it "potentially places Utah citizens at risk."
• Citing safety guardrails built into the agreement, the state agencies involved declined to suspend the program.

Where it stands: It's a foregone conclusion that providers will offload much of their administrative taskload onto AI bots. What's still up in the air is how much actual health care will be outsourced, as well.

• There's growing evidence that AI systems can do some things just as well as, if not better than, human doctors.
• A Mayo Clinic study published this week found that an AI model detected early signs of pancreatic cancer — one of the deadliest cancers — up to three years before human specialists.
• The federal government itself is experimenting with using clinical AI agents to provide around-the-clock cardiovascular care.

The other side: One argument for letting AI perform tasks currently limited to licensed professionals is that a lot of Americans simply don't have access to those professionals.

• Citing physician shortages and nationwide health costs, Doctronic's co-founders and co-CEOs pushed back on criticism of the program in a recent blog post.
• "The greatest risk in healthcare is continuing to rely on a system that cannot meet current or future demand," they argued.
• "Expanding access requires rethinking how care is delivered, with AI playing a central role in managing routine tasks while physicians focus on complex decision-making."

Between the lines: Accepting that AI will help solve provider shortages doesn't negate the need for better regulatory systems to oversee such autonomous agents, the authors of a new JAMA Perspective piece argue.

• "The adaptive nature of that technology and the risks associated with many clinical decisions demand a fundamentally different regulatory framework," write authors Alon Bergman, Robert Wachter and Ezekiel Emanuel.
• They call for a licensing regime, similar to what's in place for doctors, saying the existing FDA framework built for drugs and devices is "ill suited to adaptive, general-purpose systems."

What we're watching: I asked Emanuel, a bioethicist and adviser to former President Biden, what this all means for human doctors.

• "I think one of our most important questions is, as a profession … what is our value-add to patients? That's what AI is posing to us," he said.
• "There is going to be some displacement, absolutely."

PitchBook recently released a provocative research report titled "AI Will Deliver Care to Billions and Break the System That Built It" that further dissects the AI-replacing-doctors discussion.

Why it matters: The report essentially outlines a U.S. health care system reimagined by AI and not only lays out who does what or how it's regulated, but also how care is paid for.

Between the lines: One of the most interesting findings to me is that AI will actually increase health care costs in the short term, because it will enable more diagnoses and drive up demand for services.

• "The massive cost burden of higher near-term AI-induced utilization (five to seven years) will bring the system to a financing breaking point with the cumulative increase in medical costs reaching $5.9 trillion by the late 2030's," the authors project.
• Costs will only go down once new models that tie payments to outcomes are put in place.
• In the meantime, reduced administrative costs through the use of AI may help ease the additional cost burden.

Also notable: The report projects humans will remain in the loop on decisions for five years while handing off tasks like refilling prescriptions, scheduling office visits and interpreting routine lab test results to automated systems.

• But within a decade, systems that make clinical decisions on their own will obtain regulatory approval for "well-defined, lower-risk scenarios," it says.

My thought bubble: If the projections are anywhere close to being right, the health care industry is in for some massive upheaval.

• There also are enormous political implications. No one is happy with the current level of health care spending, whether by the government or the private sector.
• Sure, that's been true for a long time, and everyone's just kept saying it's unsustainable. But now, there could be a change agent.