Citation-amnesia paper published

A large share of biomedical researchers fail to put their findings into context by citing related, previously published work. I termed this citation amnesia, when I wrote about it 18 months ago, based on data presented at a meeting on peer review and publishing. Readers who seek more details than my initial blog provided can now pore over the stats from that research for themselves. The Johns Hopkins University team that I encountered at the Vancouver meeting has now formally published its analysis in the January 4, 2011 Annals of Internal Medicine.

Karen Robinson and Steven Goodman analyzed 227 meta-analyses — studies that compared, contrasted and probed for trends among data gleaned from multiple previous studies. The meta-analyses had all been published in 2004 and dealt with data from 1,523 randomized, controlled human trials that had initially been released between 1963 and 2004.

The Hopkins pair assumed that any related randomized, controlled trial published at least a year earlier should have been on the radar screen of that later study’s authors. In fact, fewer than 25 percent of such RCT studies that could have — and presumably should have — been cited actually were.

Especially disturbing: Of 1,101 RCTs having at least five prior trials available to cite, 23 percent referred to none of them and another 23 percent cited only one. Even more troubling, the share of trials exhibiting amnesia remained constant over the decades even as the body of evidence available for citation accumulated.

If the later authors were truly oblivious to earlier data, they risked wasting precious research dollars to repeat the same work. They also lost an opportunity to use those data in shaping the design of their study and what it sought to test.

For instance, if an earlier study showed no overall effect of a drug on the population treated, but a non-significant trend that hinted at a possible effect in a subset of the patients — diabetics or postmenopausal women, for instance — then the later study should have focused on those sub-populations or at least included a disproportionate share of people from them to probe the potential for a therapy to selectively offer them some benefit or harm.

“An incomplete picture of preexisting evidence violates the implicit ethical contract with research participants that the information they provide is necessary and will be useful to others,” Robinson and Goodman argue. They recommend that funders, institutional review boards and journals establish rules — or at least new guidelines — to “promote better use of prior research and thereby better satisfy the ethical and scientific requirements for justifiable clinical research.”