Dec 14, 2017

AI advances at detecting cancer — but it can't see you now

Illustration: Rebecca Zisser / Axios

Medicine is poised to be one place where AI makes a mark. In a study published this week, researchers report that a machine algorithm was as good — or better — than pathologists at detecting the spread of a type of breast cancer.

For all the talk about the promise of AI radically changing medicine, this is one of the first peer-reviewed studies to back claims that algorithms can detect abnormalities in pathology slides, says Eric Topol from the Scripps Research Institute.

The bottom line: Radiologists and pathologists are likely to be the first in medicine affected by AI. But researchers working on the technologies don't see them replacing doctors, and instead aiding them. And even that role will require more data about the impact on the medical profession and whether AIs are accurate enough to diagnose patients.

“It is the early days,” Aidoc CEO Elad Walach says. “There’s not enough research at this point. Deep learning has been commoditized generally but it hasn’t been commoditized for the medical domain. The algorithms out there aren’t good enough as is. We need a lot of R&D to make AI work in this space. It is not just plug and play.”

What’s new: Babak Ehteshami Bejnordi and his colleagues from Radboud University Medical Center in the Netherlands evaluated algorithms submitted in a competition to analyze tissue samples from the lymph nodes of breast cancer patients. (Cancer cells are most likely to spread to these nearby areas first so they're involved in determining a patient's prognosis.) They then compared the accuracy of AI diagnoses with those of pathologists in two different situations where the researchers had a gold-standard test to check both:

  1. A panel of 11 pathologists had two hours to review 129 digitized images of samples from patients who had already received a diagnosis from a pathologist.
  2. One pathologist was given unlimited time to review all the cases. (The expert took 30 hours.)

The result: The top seven algorithms — all deep learning methods, which have lately seen progress in image and pattern recognition — performed better than the pathologists in identifying the metastases, but were on par with the pathologist whose time wasn’t restricted.

Keep in mind: The time constraints put — or not — on the pathologists in the study aren't the reality in which they practice. And, the AIs detected just one type of breast cancer. "We need to see it borne out across lots of other pathologies not just lymph nodes for breast cancers," says Topol. "This is the most impressive paper yet. But there are limitations. This is done in silico and is not a real world validation."

More opinions

PathAI: Andy Beck, whose team won the AI competition in the new study and who is now CEO of PathAI, says AI’s arrival to pathology will be a transformation rather than a disruption.

Seeing it as the latter “betrays a lack of understanding of how these fields operate. There are so many things physicians do. Typically an AI does one specific thing very well. We aren’t even close to doing the whole breadth of what a physician does.”

Aidoc: This Israeli startup is developing technology that can detect visual abnormalities — whether it be a cancer, stroke, bleeding or an edema — in head and neck CT scans. Their focus right now, says Walach, is on optimizing radiologists’ workflow from the current practice of reviewing cases in order received to getting AI to flag urgent ones first.

They’re currently testing their technology in 5 U.S. sites. Early, unpublished results at one hospital found that the AI could spot an abnormal scan with 98% sensitivity compared to what clinicians call the "ground truth" (in this study, the diagnosis by three radiologists working without AI), says Walach. They will look to publish their findings soon.

Walach says, “There is a need for peer reviewed publications about the outcomes not just the accuracy of these systems, and leading companies should invest time and resources in publishing clinical evidence.”

A big challenge: Like other cancer tests, there is a risk of detecting — and then treating — a cancer that isn't there. That isn't unique to machines but "algorithms are tuned to perform at maximum sensitivity, meaning there may be false positives," says Stanford University's Daniel Rubin, who develops imaging tools for radiology. “As we introduce these technologies, if people don’t improve accuracy and there are more false positives, it will increase the cost of health care.”

Go deeper: We asked four medical experts whether AI might help their profession

Go deeper

Updated 2 mins ago - Politics & Policy

George Floyd protests: What you need to know

Police block protesters at a rally on May 30 outside the state house on the fourth straight day of demonstrations against the death of George Floyd. Photo: Megan Jelinger/Anadolu Agency via Getty Images

Protests over the death of George Floyd and other police-related killings of black men spread across the U.S. Saturday, amid tense standoffs with police in several cities.

The big picture: Floyd's fatal run-in with police is the latest reminder of the disparities between black and white communities in the U.S. and comes as African Americans grapple with higher death rates from the coronavirus and higher unemployment from trying to stem its spread.

U.S. cities crack down on protests against police brutality

Photo: Megan Jelinger/Anadolu Agency via Getty Images

Major U.S. cities have implemented curfews and called on National Guard to mobilize as thousands of demonstrators gather across the nation to continue protesting the death of George Floyd.

The state of play: Hundreds have already been arrested as tensions continue to rise between protesters and local governments. Protesters are setting police cars on fire as freeways remain blocked and windows are shattered, per the Washington Post. Law enforcement officials are using tear gas and rubber bullets to try to disperse crowds and send protesters home.

Trump to invite Russia and other non-member G7 countries to summit

President Trump at Cape Canaveral, Florida, on Saturday. Photo: Saul Martinez/Getty Images

President Trump told reporters on Saturday evening he would postpone the G7 summit to September and expand the meeting to more nations that are not members of the Group of 7.

Details: Trump said he would invite Russia, South Korea, Australia and India to the summit, according to a pool report. "I don’t feel that as a G7 it properly represents what’s going on in the world. It’s a very outdated group of countries," he said.