Providence Health executive talks AI strategy and inbox pain

Alongside note-taking, Providence Health executive Sara Vaezy says there's another source of clinician fatigue ripe for an AI assist: providers' inboxes. "It's a massive problem. It causes the same amount of pain as the ambient, the documentation problem," she tells Axios.

Why it matters: As Providence's chief transformation officer, Vaezy oversees the health system's AI strategy, from small operational improvements to big transformational projects.

This Axios Expert Voices interview has been edited for length and clarity.

How have you stratified your AI strategy across the health system?

• "Very near term, we're going super deep on ambient [scribe] assistance and "in basket" management. On the ambient side, we're taking a pure partner approach — we work closely with Nuance's Dax copilot.
• We have a thesis around where that space needs to go. It's not just about creating an unstructured note. Why do we have clinicians doing coding? If there's a conversation that could be valuable for outcomes, then you could look at clinical decision support, for example."

Tell me more about "in basket" management. Do you mean inbox?

• "You and I have our email inbox. The "in basket" is the clinical person's equivalent in the electronic medical record.
• It's a really big problem — something like 15 million messages a year — from a variety of sources: patients, other physicians, payors, and other partners."

How are you approaching that issue?

• "We took it two ways: Supporting a patient so they can get their needs met without having to send a message and a clinician-facing experience that helps them more efficiently respond to messages.
• We've built two homegrown solutions that tackle message volume, message routing, and efficiency of responding to messages."

Give me an example of an AI tool you sunsetted because it didn't deliver. What did you learn?

• In the early days on the "in basket" side, folks were testing out things like auto-drafting — reading a message and auto-drafting a response. We piloted it, and our physicians didn't like the auto-drafting.
• They were rewriting [the drafts], basically. The messages didn't sound like them, maybe they weren't friendly enough, maybe it didn't fit into their operations, etc. So we turned the pilot off.
• Then our CMIO had a really interesting realization: physicians don't necessarily need something auto-drafted. What they want is to be able to respond efficiently.
• So they came up with shortcut tools that say, for example, this message is asking about how to manage a fever. [The physician] knows the patient, knows it's safe within the context of this message to recommend acetaminophen or ibuprofen, and they can pick those shortcuts, and they're turned into patient-friendly messages.
• That way the physician is a conductor, not a copy editor."

Tell me about AI initiatives you've successfully moved from from pilot to production.

• "One element of in-basket management is Grace, a chatbot that lives on our website and app and MyChart. We call it a conversational navigation platform.
• Grace can help with appointment booking, medication management, billing assistance, MyChart navigation. It can provide a kind of multiple-choice option, or do free text.
• It has about 150,000 monthly active users."

Who leads Providence's AI efforts and who is working on what?

• "We created a very small group to develop the strategy of going really deep instead of broad, doing a mix of partner and homegrown [efforts].
• We also built a group comprised of clinicians and interdisciplinary people including administrators from across the system to build guard rails, led by our health care intelligence team.
• We're also a Catholic nonprofit health care system and we signed on to the Rome Call for AI Ethics, which the Vatican released."

How big are these teams?

• "In total, it's about 200 folks.
• We have 50+ practicing physicians involved, about 10 folks from health care intelligence either building small models, or guardrail frameworks, about 20 folks building software — engineers, product managers, scientists.

How widely and how long do you deploy your pilots, generally speaking?

• "We always start with a limited amount of traffic, like 1%, and then see how things progress. Then we can scale it up — we can go up to 5%, for example — and let things stabilize, learn more.
• Sometimes we'll go back down too, or we'll go clinic-by-clinic.
• The length often depends on the statistical significance from the data we collect. We need to have at least 1,000 interactions in a use case to scale up beyond that use case. It's not just time-driven.

This is a bit of a moonshot question. Do you see a world in which a large ambient scribe company could eventually compete with the likes of an Epic or an Oracle?

• "It's that classic question of whether disruptive solutions can be so much better than an incumbent to overcome the distribution problem while the incumbents figure out how to build a better product.
• That's really hard to do. If a company can play a long game, it's possible, but it would be a decade-long process."