The era of Doctor AI is already here

Where it stands: OpenAI put out some numbers in January: More than 40 million people ask its ChatGPT health care-related questions every day, and 1 in 4 of the tool's approximately 800 million regular users submits a health care prompt every week.

• The careful debate over how AI should be deployed, regulated and evaluated in clinical settings often fails to acknowledge that the cat's already out of the bag when it comes to direct-to-consumer use.
• "Too often people are using this as an expert and not as an assistant," American Medical Association CEO John Whyte told Axios in an interview.

Between the lines: Everyone pretty much agrees that you shouldn't replace your doctor with AI, at least not yet. But a more realistic question is how helpful it is when your doctor isn't available — or if you don't have one.

• "We've made accessibility to medical information and medical judgment so hard in this country, and ChatGPT makes it so easy," said Ashish Jha, the former White House COVID response coordinator under President Biden and former dean of the Brown University School of Public Health.
• "The idea that these tools have to be as good as a physician is absurd given how much more convenient they are."
• "I think there's a risk of bad things happening. ... Is it dangerous? I think the status quo is dangerous," said Bob Wachter, chair of the Department of Medicine at UCSF and the author of "A Giant Leap: How AI is Transforming Healthcare and What That Means for Our Future."
• "The question is without it, what would you have done?" Wachter added.

Driving the news: A recent study published in Nature found that ChatGPT under-triaged about half of health care emergencies in a test performed by researchers.

• Karan Singhal, who leads the company's health AI team, said its latest GPT-5 models correctly refer emergency cases nearly 99% of the time. In real life, she said, health conversations in ChatGPT typically unfold over multiple turns, where the model asks follow-up questions and gathers more context before responding.

What we're watching: What new state and federal guardrails are put around AI in health care.

• "We don't regulate the availability of information in the United States," said David Blumenthal, former president of the Commonwealth Fund, but "it's possible that rating agencies may arise that will address the reliability of different chatbots for different functions."

Some takeaways from my conversations with experts:

1. AI seems to be better at some things than others. Chatbots can be good at explaining lab results or coming up with a list of questions to ask your doctor ahead of a visit, Whyte said.

• That doesn't mean people are actually using it for what it's good at.
• Jha, who said that large language models aren't yet "ready for prime time" when it comes to diagnosing illness, still thinks people will use it for clues to what ails them "because they've been using Google for diagnosis and this is so much better than Google."

• Ultimately, "I don't think we have a super clear understanding of what it's good for and what it's not," Jha said.

2. Output is super dependent on input. And your average person may not know the correct inputs.

• "The way a patient's question can be phrased can lead to variability in how an LLM responds," said Duke University's Monica Agrawal.
• "If they have incomplete context or they share a subjective impression or they have a misconception when they're seeking advice, LLMs have an ability more so than a doctor to reinforce those misconceptions."

3. The way it says things can be problematic. "I worry some of these LLMs speak with a level of confidence that is really unjustified," Jha said.

• It is also problematic that models generally are built to tell people what they want to hear, Agrawal said. "In the places where a doctor might push back ... we're not seeing necessarily the same behavior in models."
• "If you say, 'I have a headache,' I don't say, 'Oh I think you have a migraine' — I would say, 'Tell me more about it,'" Wachter said. "The tools don't naturally do that, and I think the consumer-facing tools of the future will."

4. Most people using AI don't have the expertise to spot mistakes. There's a divide between "professional use of these tools and the laypeople use of these tools," Wachter said.

• Whereas they can be extremely helpful to doctors, your average patient probably doesn't have the medical knowledge to identify when a response doesn't apply or seems off.

What we're watching: Today's models are constantly being re-trained — and generally improved.