Discrepancies common between reported medical outcomes, trial registry data

Only a quarter of publications reporting on headache clinical trials were registered in an approved clinical trial registry, a new study published today in Neurology has found.

That’s true, despite a 2005 decision by major medical journals that they would only consider for publication results of clinical trials that had been registered in an approved clinical trial registry, such as clinicaltrials.gov. The goal was to reduce selective reporting of clinical trials, where researchers publish results that are outside the scope of the study’s original design and, thus, cannot be considered scientifically accurate.

“From 2005 to 2014, we found that only 26 percent of randomized trials published in core headache journals were compliant with trial registration requirements and that 38 percent of registered trials published results that did not match what authors initially planned to report,” explains Melissa Rayhill, lead author, assistant professor of neurology in the Jacobs School of Medicine and Biomedical Sciences and a UBMD Neurology physician specializing in headache medicine.

“Our study suggests that selective reporting remains a problem in the headache medicine literature and should bring into question the quality of similar reporting in other disciplines throughout the medical literature," she says.

Rayhill became interested in potential pitfalls of the peer review process while serving as an ad hoc reviewer for several journals during her fellowship training.

“Although rigorous, if the few people who are asked to review a paper fail to bring up a potential bias or study design flaw, it may potentially be published without addressing the issue at all," she says. “Whether or not the paper answers a question the study was designed to address is critical, but that issue may not even be considered in the peer review process. In addition, reviewers are working to meet deadlines and often receive no compensation or inadequate compensation for their time.

“These realities led me to more closely scrutinize the quality of data available in the literature from which we develop clinical guidelines and guide individual clinical decisions.”

Rayhill and her co-authors conducted their comprehensive study of every trial published between 2005 and 2014 in the top journals in the headache medicine field: Headache, Cephalagia and the Journal of Headache and Pain. A total of 225 randomized controlled trials were studied. In 26 percent of them, a trial registration number was reported that could be linked to a corresponding registry.

The authors note the frequency of registering clinical trials has increased in recent years in the headache medicine literature and that percentages range from comparable to somewhat higher in other fields, such as geriatrics and surgery.

“I hope our findings can positively impact the medical research process, increase awareness of trial registration and its rationale, and that it may indirectly improve the quality of data available to physicians to make good clinical decisions for patients,” Rayhill says.

The UB Department of Neurology and the UBMD Neurology physician practice plan hired Rayhill earlier this year to start a headache medicine practice. She came to UB after completing fellowship training in headache medicine at Harvard Medical School/Brigham and Women’s Hospital.  

The study, which was unfunded, was conducted while Rayhill was a fellow at Harvard. Her co-authors at Brigham and Women’s Hospital and Harvard Medical School are Rebecca Burch, instructor of neurology; Elizabeth Loder, chief, Division of Headache and Pain, and associate professor of neurology; and Roni Sharon, former fellow at the hospital’s John R. Graham Headache Center.

Neurology is the medical journal of the American Academy of Neurology.