May 11, 2017

How economists want to ease science's replication crisis

Lazaro Gamio / Axios

Economists Luigi Butera and John List at the University of Chicago recently developed a new approach to encourage researchers to replicate experiments.

Their idea: Researchers conduct their study, write it up, and then publish it online rather than in a journal. The deal is they commit to not sending it to a peer review journal initially and instead will offer collaboration with people who are willing to repeat it. Afterwards, they all publish one paper together in a journal.

Why it matters: At the heart of science is a process of validating a discovery or finding by someone else replicating an experiment and producing similar results. But there's mounting, if at times contentious, evidence that science has a replication problem. Opinions vary, of course, but no one disputes that the act of replication is important. However, incentives are low since top journals and journalists have a taste for novel studies, so there is little glory in going second. Plus, it can be difficult to secure funding for follow-up studies.

Another perspective: More than 20 years ago, Gary King suggested researchers should make their data available when they publish results so other researchers can try to replicate their findings — which is now a relatively common practice. Axios spoke to King about the economists' new proposal, and he said the scheme is clever but has some issues. For example, he said, it is unlikely famous professors would replicate the work of graduate students.

The bigger picture: "The point of science is you can't learn from one article. You learn from a community of people working in cooperation and competition. I could never trust one article," King said.

People are highly influenced by their prior beliefs. The University of Chicago economists found, for example, that if you ask two scholars how likely they will believe a novel finding that bucks convention, both will be very skeptical initially. If you show them a study, they will update their beliefs. The first who initially thought there was a 1% chance, then said it's 13%. The second jumped from 10% to 64%. Butera said a few (in their study, three) successful replications will cause people's beliefs to converge on an even higher number (80%).

And, when a result can't be replicated? "I don't see it as a problem. It is the definition of what has produced all of the progress we've seen over the last 400 years," King said.

Go deeper

Pentagon watchdog sidelined by Trump resigns

Fine testiying before the Senate Judiciary Committee in 2017. Photo: Drew Angerer/Getty Images

Glenn Fine, the Pentagon's principal deputy inspector general, submitted his resignation on Tuesday.

Why it matters: President Trump removed Fine as the Pentagon's acting inspector general in April 7 after a group of independent federal watchdogs selected him to lead the Pandemic Response Accountability Committee, which was set up to oversee the rollout of the $2 trillion coronavirus relief bill.

Coronavirus dashboard

Illustration: Aïda Amer/Axios

  1. Global: Total confirmed cases as of 12 p.m. ET: 5,534,728 — Total deaths: 347,587 — Total recoveries — 2,260,945Map.
  2. U.S.: Total confirmed cases as of 12 p.m. ET: 1,667,154 — Total deaths: 98,371 — Total recoveries: 379,157 — Total tested: 14,604,942Map.
  3. 2020: Trump pushes for a polarized pandemic election.
  4. States: Michigan Gov. Gretchen Whitmer says "I can’t for the life of me understand" Trump’s antagonism toward her state — New York reports lowest number of new coronavirus deaths since March.
  5. Public health: The final data for remdesivir is in and its benefits are rather limited.
  6. Education: A closer look at how colleges can reopenNotre Dame president says science alone "cannot provide the answer" to reopening.
  7. What should I do? Hydroxychloroquine questions answeredTraveling, asthma, dishes, disinfectants and being contagiousMasks, lending books and self-isolatingExercise, laundry, what counts as soap — Pets, moving and personal healthAnswers about the virus from Axios expertsWhat to know about social distancingHow to minimize your risk.
  8. Other resources: CDC on how to avoid the virus, what to do if you get it, the right mask to wear.

Subscribe to Mike Allen's Axios AM to follow our coronavirus coverage each morning from your inbox.

Updated 25 mins ago - Politics & Policy

New York reports lowest number of daily coronavirus deaths since March

The number of daily new coronavirus cases and deaths reported in New York was the lowest since the state started its lockdown in March, Gov. Andrew Cuomo said Tuesday, calling Memorial Day a "pivot point" for New York.

By the numbers: 73 New Yorkers died from coronavirus in the past 24 hours, and 200 people tested positive. Hospitalizations and intubations also decreased.