Cancer data: Burying bad news

When doctors are looking to treat an illness, they often rely on reports in medical journals. Sometimes those reports describe striking successes with a particular therapy. Less frequently, they highlight failures. To physicians, learning what doesn’t work or has troubling side effects can be as important as knowing which therapies hit a home run.

Yet doctors — and the reporters who highlight research news — are getting a very skewed picture of research findings if they rely on what makes it into major medical journals.

The reason: Data from fewer than one in five research trials are ever published. Findings from the vast majority of human trials become buried for reasons that may never come to light, according to a new study in The Oncologist. It’s published early and online September 24.

For the past nine years, the National Institutes of Health have maintained a registry of medical trials. Researchers must list theirs with this ClinicalTrials.gov registry. If they don’t, major medical journals (those that belong to the International Committee of Medical Journal Editors) will not publish the trial’s findings.

Scott Ramsey and John Scoggins of the University of Washington, Seattle, scoured that registry for any trial that was supposed to have been conducted to gauge the effectiveness of cancer treatments — and turned up 2,028. The pair then cross-checked these trials against all published studies that were listed in PubMed, a comprehensive database compiled by the National Library of Medicine. PubMed includes citations to all medical and related studies that have been published in peer-reviewed journals.

Among the cancer trials that were identified as completed or halted as of a year ago, just 17.6 percent were ultimately published in PubMed-listed journals, Ramsey and Scoggins report.

Okay, maybe the halted trials were pulled because of what turned out to be poor research design, failure to recruit enough patients, or some other reasonable issue. But even after restricting the analysis to only those trials that were completed, the share that ended up being published still amounted to fewer than one in five.

Who conducted a trial seemed to influence the likelihood its data would see the light of day. Studies sponsored by industry (such as drug companies) had the lowest publication rate: 5.9 percent. By comparison, data from 59 percent of studies performed by clinical-trial networks were published.

Ramsey and Scoggins turned up 341 cancer trials from the registry that were published. Of these, two-thirds reported positive — expected and beneficial — findings.

The new analysis raises the ugly specter of publication bias, its authors say. The assumption is that many if not most unpublished trials involved treatments that didn’t work. “Of particular concern,” they argue, is the especially poor showing by industry-sponsored trials, since they tended to probe the value of patented drugs — “many of which are in clinical use.”

James H. Doroshow, director of the National Cancer Institute’s division of treatment and diagnosis, notes that last year alone some 50,000 patients took part in trials that his institute funded. The “apparent lack of access to the final efficacy and toxicity data for cancer clinical trials from all sponsors, but especially for industry-sponsored studies, poses multiple scientific and ethical questions,” he charges in an editorial accompanying the new paper.

For instance, as doctors begin developing novel chemotherapy cocktails — mixtures of drugs initially tested on their own — toxic reactions may emerge. It’s imperative, Doroshow says, that inklings of such side effects be communicated immediately “to the entire oncology community in the peer-reviewed literature.”

Moreover, he notes that some drug-safety trials did not publish their findings, or did not do so early enough, such that they could inform subsequent trials. This practice is not likely to continue, he points out, since new federal rules will fine investigators who fail to post outcome data for all trials getting money from Uncle Sam. Moreover, for cancer trials, NCI will require that researchers begin reporting outcome data — treatment successes and failures — throughout the course of the trial, not just at the end.

A second editorial, this one by The Oncologist’s senior editor, Gregory A. Curt (an employee of drug company AstraZeneca), and editor-in-chief Bruce A. Chabner (of Harvard Medical School) find the new analysis by Ramsey and Scoggins “thought-provoking and disturbing.” At a minimum, they argue, publication of trial data should be considered “an obligation” for any researchers recruiting patients who contribute “their precious time and well-being, and for some, their very lives.”

This is especially true for drug-company trials, they contend, since “industry has become the dominant sponsor of new drug trials.”

But Curt and Chabner also suspect that part of the problem illustrated in the new paper traces to issues other than a drug company’s interest in hiding bad data.

For instance, study authors “face the hurdle of finding a journal willing to publish a negative, poorly designed or inadequately accruing trial.” This is especially true, they say, since “Journals live and die based on their Impact Factor” — how often they’re cited by subsequent papers. Any journal filled with such findings would “not attract readership, citations and advertisement,” they write.

One solution Curt and Chabner propose: Make NCI funding for new trials dependent on the investigators’ past track record of getting their trials data — both positive and negative — published. They also argue that there’s “a need for a new venue” to record outcomes of well-executed but ultimately negative clinical trials. This database must be searchable via PubMed and other search engines. Currently, these editors note, The Oncologist is considering whether it should become a repository for such cancer data.

Bottom line: It should become increasingly harder for drug companies and others to bury embarrassing findings. But the impacts of coming changes might not show up for five years or more. In the mean time, let’s hope our research-funding agencies, our hospitals and our doctors don’t assume that when it comes to trial data, no news is good news.