Axios House: Trust presents the biggest challenge in health care AI, industry leaders say

DAVOS, Switzerland — Leaders in the health care and tech space said transparency, feedback and narrative play a crucial role in building provider and patient trust in AI, during a Jan. 20 Axios event.

Why it matters: Health care AI has the potential to help improve patient outcomes and lower the risk of physician burnout.

Axios' Courtenay Brown moderated the Expert Voices conversation, which was sponsored by Philips.

What they're saying: Trust in health care AI is particularly low, said Vanessa Parli, managing director of programs and external engagement at the Stanford Institute for Human-Centered AI (HAI).

• Using the word "productivity" about AI can be a deterrent to providers because it may imply less time seeing patients, according to Eric Cioè Peña, Northwell Health vice president and the founding director of its Center for Global Health.
• "Trust is a narrative problem first, and a technology problem second," added Hugh Taggart, CEO of Weber Shandwick's EMEA region.

By the numbers: A survey from Philips found a gap between providers (63%) and patients (48%) when it came to optimism about AI improving patient outcomes.

Yes, but: Keeping humans in the loop has been key to successful implementation strategies at Mayo Clinic, said Matthew Callstrom, physician lead of AI and strategy at Mayo Clinic.

• "When you start to explain to patients how we're using AI, they're like, put me in," Callstrom said.
• This has helped on the provider side as well, he said. For instance, a volunteer innovation team at Mayo built and tested an AI-driven handoff tool for the transition between nurse shifts. It has since been adopted by 75% of the nurses, generating hundreds of thousands of summaries.

What's next: The industry should "think about it in terms of what we know, from research, drives trust: accountability [and] transparency," American Psychological Association CEO Arthur Evans said.

• Anthropic's Claude for Healthcare is "building connectors to key data sources that we know are trusted, like the CMS coverage database or the National Provider identification registry," said Elizabeth Kelly, head of beneficial deployments.
• "The systems should be very forthcoming with what they will not do," Peloton CTO Francis Shanahan said.
• "You need to design [the technology] in a way so that the consumer understands what the implications of the technology are," Parli added.

Content from the sponsor's remarks:

"Three out of every four patients say they wait over two months for specialty care," Philips chief innovation officer Shez Partovi said. "Of those, a third of them — while they're waiting — end up in a hospital."

• "We're putting AI into everything … from toothbrushes to MRI scanners and angio equipment, because it actually helps automate the procedure and … improve access."