By Janet Raloff
VANCOUVER, Canada — Peer reviewers for biomedical journals preferentially rate manuscripts with positive health outcomes as better, a new study reports.
The findings caused a buzz when presented September 11 at the International Congress on Peer Review and Biomedical Publication. If positive trials are preferentially published, explains Seth Leopold of the University of Washington Medical Center in Seattle, doctors will get a skewed impression of a therapy’s value: “Novel treatments will appear more effective than they actually are.”
To test whether journals give negative or equivocal findings short shrift—despite pledging not to—Leopold’s team asked more than 200 trained reviewers at two orthopedic journals to rate whether manuscripts were worthy of publication. The team included a bogus manuscript in two forms. Data in the first showed better prevention of infection by one of two antibiotic regimens. In the second version, neither treatment outperformed the other but the results still would have affected patient care. The papers were identical, except for the outcomes.
Among the 55 reviewers at one journal who were asked to evaluate the positive-outcome manuscript, 98 percent recommended that the journal publish it. Only 71 percent of another 55 reviewers from the journal who got the no-difference paper rated it ready for prime time. A similar, though not statistically significant, trend emerged at the second journal.
Readers at both journals gave the positive paper’s method section higher ratings, even though the other paper had identical methods. And readers of the positive paper were less likely to spot intentionally included mistakes, Leopold reported.
NOTE: To find out more, read a longer Science & the Public blog entry by Janet Raloff on this topic posted on September 11 and available
here
.
