Science’s self-criticism makes the enterprise stronger

Ever since John Ioannidis published his influential 2005 essay about the statistical problems plaguing science, there has been deepening concern about the enterprise. Are most scientific findings wrong? Have quality control and informed skepticism given way to publish-or-perish and headline grabbing? If science is self-correcting, why isn’t it working?

In response to such concerns, many scientists (and scientific institutions) have become more self-reflective about how science works and how it might be improved, as Tina Hesman Saey reports in “Is redoing scientific research the best way to find truth?” In the first of a two-part series, Saey describes efforts to look specifically at one part of science’s self-correction system: the ability to reproduce experimental findings. That’s at the core of the scientific process. But it often doesn’t work. Saey examines the reasons it sometimes fails and discusses possible solutions (such as hiring a company to repeat your findings for you, or requiring more detailed descriptions of methods). She also describes an emerging argument about what scientific reproducibility really means. Some scientists believe that rather than actually replicating an experiment, step by step, it’s more important to use a different experiment or approach to study the same question. From that perspective, current efforts to duplicate a study exactly may have less value than simply expanding research on any given issue. Because so many variables influence life and psychological sciences, this way of thinking about the problem may be especially important in these fields, Saey says.

Raising these issues poses another problem: Criticizing the scientific process can be seen as giving an opening to antiscience factions looking for reasons to plant doubt in the public mind. Scientific infighting looks bad from the outside. But self-criticism and self-skepticism have always been part of good science. Discussing the valid concerns without fueling the “all science is wrong” attitude is difficult. I think Saey does an admirable job of explaining the scope of the problem, and exploring responses to it. And she offers enough perspective to show the importance of acknowledging science’s problems without exaggerating them. In the end, a dose of self-reflection, instead of shaking science’s core, can help make it more robust.