The high stakes of low scientific standards
- Miriam Kramer, author of Axios Space
Illustration: Aïda Amer/Axios
In the midst of this pandemic, science is suffering from low standards for some research, a new study argues.
The big picture: Science — which is slow, methodical and redundant — isn't necessarily made for the immediacy and acute public interest brought on by a health crisis.
- Scientists rely on peer review and back and forth exchange that leads to a more polished final study. But a health crisis like the current pandemic, or the Ebola outbreak, creates a sense of urgency that can be antithetical to the scientific process.
What's happening: A new study out today in the journal Science warns many of the clinical trials and studies first published about treatments and other issues involving the current pandemic were designed poorly or had other issues that affected their outcomes.
- Studies that have yet to go through peer-review — like a recent, flawed study of the use of hydroxychloroquine to treat coronavirus — have found their way into news stories thanks to pre-print services, leading to problematic reporting and real-time peer review through Twitter.
- More than 18 clinical trials testing hydroxychloroquine to treat the novel coronavirus have enrolled more than 75,000 patients in North America.
- "This massive commitment concentrates resources on nearly identical clinical hypotheses, creates competition for recruitment, and neglects opportunities to test other clinical hypotheses," the study says.
- Early, flawed work has potentially increased the risk that later results may have gotten false positives and more media attention than they deserved, the new study says.
Yes, but: While the pandemic is exacerbating these problems with misinformation and lax research standards, it isn't the cause of them.
- "Some of the problems that we're seeing right now are actually not that exceptional compared to the problems that we have under normal conditions as well, just that maybe they're a little bit more amplified and have a little more visibility," Jonathan Kimmelman, director of the Biomedical Ethics Unit at McGill University and one of the authors of the new paper, told Axios.
- These kinds of issues cropped up during previous health crises, and while the authors of the new study argue that some of those problems around information sharing and standards of research have improved, there's still a long way to go.
What's next: Many of these issues around varying standards of research and communication could be remedied through better communication among researchers and the agencies funding their work.
- Instead of having a number of fragmented studies competing for resources and looking for effective treatments, the researchers say it would make more sense to bring them under one umbrella, allowing them to coordinate.
- "You could reduce variation, and you might get answers more quickly," Alex John London, the director of the Center for Ethics and Policy at Carnegie Mellon and one of the authors of the new study, told Axios.
- The authors are also calling on clinicians to resist performing their own small studies, instead opting to join up with larger trials.
- They also say agencies need to help build those larger studies and avoid making statements to the public about unvalidated treatments that may or may not work, instead opting to elevate larger studies in their various stages to the public.
Go deeper: