Dec 14, 2017

AI advances at detecting cancer — but it can't see you now

Illustration: Rebecca Zisser / Axios

Medicine is poised to be one place where AI makes a mark. In a study published this week, researchers report that a machine algorithm was as good — or better — than pathologists at detecting the spread of a type of breast cancer.

For all the talk about the promise of AI radically changing medicine, this is one of the first peer-reviewed studies to back claims that algorithms can detect abnormalities in pathology slides, says Eric Topol from the Scripps Research Institute.

The bottom line: Radiologists and pathologists are likely to be the first in medicine affected by AI. But researchers working on the technologies don't see them replacing doctors, and instead aiding them. And even that role will require more data about the impact on the medical profession and whether AIs are accurate enough to diagnose patients.

“It is the early days,” Aidoc CEO Elad Walach says. “There’s not enough research at this point. Deep learning has been commoditized generally but it hasn’t been commoditized for the medical domain. The algorithms out there aren’t good enough as is. We need a lot of R&D to make AI work in this space. It is not just plug and play.”

What’s new: Babak Ehteshami Bejnordi and his colleagues from Radboud University Medical Center in the Netherlands evaluated algorithms submitted in a competition to analyze tissue samples from the lymph nodes of breast cancer patients. (Cancer cells are most likely to spread to these nearby areas first so they're involved in determining a patient's prognosis.) They then compared the accuracy of AI diagnoses with those of pathologists in two different situations where the researchers had a gold-standard test to check both:

  1. A panel of 11 pathologists had two hours to review 129 digitized images of samples from patients who had already received a diagnosis from a pathologist.
  2. One pathologist was given unlimited time to review all the cases. (The expert took 30 hours.)

The result: The top seven algorithms — all deep learning methods, which have lately seen progress in image and pattern recognition — performed better than the pathologists in identifying the metastases, but were on par with the pathologist whose time wasn’t restricted.

Keep in mind: The time constraints put — or not — on the pathologists in the study aren't the reality in which they practice. And, the AIs detected just one type of breast cancer. "We need to see it borne out across lots of other pathologies not just lymph nodes for breast cancers," says Topol. "This is the most impressive paper yet. But there are limitations. This is done in silico and is not a real world validation."

More opinions

PathAI: Andy Beck, whose team won the AI competition in the new study and who is now CEO of PathAI, says AI’s arrival to pathology will be a transformation rather than a disruption.

Seeing it as the latter “betrays a lack of understanding of how these fields operate. There are so many things physicians do. Typically an AI does one specific thing very well. We aren’t even close to doing the whole breadth of what a physician does.”

Aidoc: This Israeli startup is developing technology that can detect visual abnormalities — whether it be a cancer, stroke, bleeding or an edema — in head and neck CT scans. Their focus right now, says Walach, is on optimizing radiologists’ workflow from the current practice of reviewing cases in order received to getting AI to flag urgent ones first.

They’re currently testing their technology in 5 U.S. sites. Early, unpublished results at one hospital found that the AI could spot an abnormal scan with 98% sensitivity compared to what clinicians call the "ground truth" (in this study, the diagnosis by three radiologists working without AI), says Walach. They will look to publish their findings soon.

Walach says, “There is a need for peer reviewed publications about the outcomes not just the accuracy of these systems, and leading companies should invest time and resources in publishing clinical evidence.”

A big challenge: Like other cancer tests, there is a risk of detecting — and then treating — a cancer that isn't there. That isn't unique to machines but "algorithms are tuned to perform at maximum sensitivity, meaning there may be false positives," says Stanford University's Daniel Rubin, who develops imaging tools for radiology. “As we introduce these technologies, if people don’t improve accuracy and there are more false positives, it will increase the cost of health care.”

Go deeper: We asked four medical experts whether AI might help their profession

Go deeper

Coronavirus dashboard

Illustration: Sarah Grillo/Axios

  1. Global: Total confirmed cases as of 1 p.m. ET: 691,867 — Total deaths: 32,988 — Total recoveries: 146,613.
  2. U.S.: Leads the world in cases. Total confirmed cases as of 1 p.m. ET: 125,433 — Total deaths: 2,201 — Total recoveries: 2,612.
  3. Federal government latest: The first federal prisoner to die from coronavirus was reported from a correctional facility in Louisiana on Sunday.
  4. Public health updates: Fauci says 100,000 to 200,000 Americans could die from virus.
  5. State updates: Louisiana governor says state is on track to exceed ventilator capacity by end of this week — Cuomo says Trump's mandatory quarantine comments "really panicked" people
  6. World updates: Italy reported 756 new deaths, bringing its total 10,779. Spain reported almost 840 dead, another new daily record, bringing its total to over 6,500.
  7. What should I do? Answers about the virus from Axios expertsWhat to know about social distancingQ&A: Minimizing your coronavirus risk
  8. Other resources: CDC on how to avoid the virus, what to do if you get it.

Subscribe to Mike Allen's Axios AM to follow our coronavirus coverage each morning from your inbox.

Cuomo: Trump's mandatory quarantine comments "really panicked people"

New York Gov. Andrew Cuomo said in a press conference Sunday that President Donald Trump's unexpected Saturday announcement of a possible "short-term" quarantine of New York, New Jersey and parts of Connecticut to curb the spread of the coronavirus "really panicked people."

Why it matters: Though Trump ruled out the mandatory quarantine later that day, Cuomo said people still called "all night long" asking about the comments and many likely fled the New York area — possibly spreading the virus further.

U.S. coronavirus updates: Fauci suggests death toll could top 100,000

Data: The Center for Systems Science and Engineering at Johns Hopkins; Map: Andrew Witherspoon/Axios

Anthony Fauci, director of the National Institute of Allergy and Infectious Diseases, said on CNN Sunday that models suggest COVID-19 will infect millions of Americans and could kill 100,000–200,000, though he stressed that the projections are "such a moving target."

The big picture: With more than 121,000 people infected, the U.S. has the most COVID-19 cases in the world, exceeding China and Italy, per data from Johns Hopkins. A second wave of American cities, including Boston, Detroit, New Orleans and Philadelphia, reported influxes of cases on Saturday.

Go deeperArrowUpdated 2 hours ago - Health