How economists want to ease science's replication crisis

Economists Luigi Butera and John List at the University of Chicago recently developed a new approach to encourage researchers to replicate experiments.

Their idea: Researchers conduct their study, write it up, and then publish it online rather than in a journal. The deal is they commit to not sending it to a peer review journal initially and instead will offer collaboration with people who are willing to repeat it. Afterwards, they all publish one paper together in a journal.

Why it matters: At the heart of science is a process of validating a discovery or finding by someone else replicating an experiment and producing similar results. But there's mounting, if at times contentious, evidence that science has a replication problem. Opinions vary, of course, but no one disputes that the act of replication is important. However, incentives are low since top journals and journalists have a taste for novel studies, so there is little glory in going second. Plus, it can be difficult to secure funding for follow-up studies.

Another perspective: More than 20 years ago, Gary King suggested researchers should make their data available when they publish results so other researchers can try to replicate their findings — which is now a relatively common practice. Axios spoke to King about the economists' new proposal, and he said the scheme is clever but has some issues. For example, he said, it is unlikely famous professors would replicate the work of graduate students.

The bigger picture: "The point of science is you can't learn from one article. You learn from a community of people working in cooperation and competition. I could never trust one article," King said.

People are highly influenced by their prior beliefs. The University of Chicago economists found, for example, that if you ask two scholars how likely they will believe a novel finding that bucks convention, both will be very skeptical initially. If you show them a study, they will update their beliefs. The first who initially thought there was a 1% chance, then said it's 13%. The second jumped from 10% to 64%. Butera said a few (in their study, three) successful replications will cause people's beliefs to converge on an even higher number (80%).

And, when a result can't be replicated? "I don't see it as a problem. It is the definition of what has produced all of the progress we've seen over the last 400 years," King said.