AI advances at detecting cancer — but it can't see you now

Illustration: Rebecca Zisser / Axios

Medicine is poised to be one place where AI makes a mark. In a study published this week, researchers report that a machine algorithm was as good — or better — than pathologists at detecting the spread of a type of breast cancer.

For all the talk about the promise of AI radically changing medicine, this is one of the first peer-reviewed studies to back claims that algorithms can detect abnormalities in pathology slides, says Eric Topol from the Scripps Research Institute.

The bottom line: Radiologists and pathologists are likely to be the first in medicine affected by AI. But researchers working on the technologies don't see them replacing doctors, and instead aiding them. And even that role will require more data about the impact on the medical profession and whether AIs are accurate enough to diagnose patients.

“It is the early days,” Aidoc CEO Elad Walach says. “There’s not enough research at this point. Deep learning has been commoditized generally but it hasn’t been commoditized for the medical domain. The algorithms out there aren’t good enough as is. We need a lot of R&D to make AI work in this space. It is not just plug and play.”

What’s new: Babak Ehteshami Bejnordi and his colleagues from Radboud University Medical Center in the Netherlands evaluated algorithms submitted in a competition to analyze tissue samples from the lymph nodes of breast cancer patients. (Cancer cells are most likely to spread to these nearby areas first so they're involved in determining a patient's prognosis.) They then compared the accuracy of AI diagnoses with those of pathologists in two different situations where the researchers had a gold-standard test to check both:

  1. A panel of 11 pathologists had two hours to review 129 digitized images of samples from patients who had already received a diagnosis from a pathologist.
  2. One pathologist was given unlimited time to review all the cases. (The expert took 30 hours.)

The result: The top seven algorithms — all deep learning methods, which have lately seen progress in image and pattern recognition — performed better than the pathologists in identifying the metastases, but were on par with the pathologist whose time wasn’t restricted.

Keep in mind: The time constraints put — or not — on the pathologists in the study aren't the reality in which they practice. And, the AIs detected just one type of breast cancer. "We need to see it borne out across lots of other pathologies not just lymph nodes for breast cancers," says Topol. "This is the most impressive paper yet. But there are limitations. This is done in silico and is not a real world validation."

More opinions

PathAI: Andy Beck, whose team won the AI competition in the new study and who is now CEO of PathAI, says AI’s arrival to pathology will be a transformation rather than a disruption.

Seeing it as the latter “betrays a lack of understanding of how these fields operate. There are so many things physicians do. Typically an AI does one specific thing very well. We aren’t even close to doing the whole breadth of what a physician does.”

Aidoc: This Israeli startup is developing technology that can detect visual abnormalities — whether it be a cancer, stroke, bleeding or an edema — in head and neck CT scans. Their focus right now, says Walach, is on optimizing radiologists’ workflow from the current practice of reviewing cases in order received to getting AI to flag urgent ones first.

They’re currently testing their technology in 5 U.S. sites. Early, unpublished results at one hospital found that the AI could spot an abnormal scan with 98% sensitivity compared to what clinicians call the "ground truth" (in this study, the diagnosis by three radiologists working without AI), says Walach. They will look to publish their findings soon.

Walach says, “There is a need for peer reviewed publications about the outcomes not just the accuracy of these systems, and leading companies should invest time and resources in publishing clinical evidence.”

A big challenge: Like other cancer tests, there is a risk of detecting — and then treating — a cancer that isn't there. That isn't unique to machines but "algorithms are tuned to perform at maximum sensitivity, meaning there may be false positives," says Stanford University's Daniel Rubin, who develops imaging tools for radiology. “As we introduce these technologies, if people don’t improve accuracy and there are more false positives, it will increase the cost of health care.”

Go deeper: We asked four medical experts whether AI might help their profession

What's next

Pelosi slams McConnell trial rules as "deliberately designed to hide the truth"

Photo: Mark Wilson/Getty Images

House Speaker Nancy Pelosi (D-Calif.) claimed in a statement Tuesday that the rules Senate Majority Leader Mitch McConnell (R-Ky.) has proposed for President Trump's impeachment trial diverge from the Clinton precedent and show he has "chosen a cover-up" over a fair trial.

Context: McConnell made public an organizing resolution Monday laying out the terms for the trial, which include 24 hours over two days for each side to present their cases. It would block evidence discovered in the House impeachment investigation from being presented without a separate vote, and would delay a vote on whether to subpoena witnesses and documents until later in the trial.

Setting the scene for Super Bowl LIV

Illustration: Sarah Grillo/Axios

After a grueling four months of football, Patrick Mahomes has led the Chiefs out of the darkness and into the Miami sun, where the 49ers football machine awaits them in the Super Bowl.

"The Gunslinger"

Mahomes is 24 years old, arguably the best player in football and he just toppled Tom Brady as the top seller of NFL merchandise. A star has already been born — now he has the biggest stage in sports to showcase his brilliance.

Go deeperArrow2 hours ago - Sports

Exclusive: The Athletic raises $50 million

Adam Hansmann (left) and Alex Mather (right), co-founders of The Athletic. Photo: Steph Gray, courtesy of The Athletic

The Athletic, a subscription-based digital sports media company, has raised $50 million in a Series D funding round, executives tell Axios.

  • With this investment, the company has raised a total of $139.5 million since its launch in 2016 and is valued at roughly $500 million after the new raise, according to sources familiar with the deal.
Go deeperArrow2 hours ago - Media