Cookies on this website

We use cookies to ensure that we give you the best experience on our website. If you click 'Accept all cookies' we'll assume that you are happy to receive all cookies and you won't see this message again. If you click 'Reject all non-essential cookies' only necessary cookies providing core functionality such as security, network management, and accessibility will be enabled. Click 'Find out more' for information on how to change your cookie settings.

The statistical analysis and reporting of treatment effects in reports of randomised trials with a binary primary endpoint requires substantial improvement, suggests NDORMS research published in BMC Medicine.

Evaluation checklist © Shutterstock

 

Clinical trials help doctors decide between different treatments and ensure that patients receive the best care possible. Good decision-making is only possible if these trials are done correctly and if doctors have access to all of the relevant information about these trials in their resulting academic publications.

A research team from the Oxford Clinical Trials Research Unit (OCTRU) and Centre for Statistics in Medicine, NDORMS, has found evidence that trials with binary outcomes are particularly badly reported. Around half of all clinical trials use binary outcomes. These trials look at two possible outcomes rather than a continuum, such as failure or success of a treatment, or whether patients are dead or alive at the end of the trial.

The researchers looked at 200 academic publications of clinical trials published in January 2019 that used a binary outcome as the main finding of the study. They examined the statistical methods used to analyse the data and whether the publication merely assessed if there was a difference between the treatment groups or included details of how big this difference was and how it was calculated. The researchers also measured how much missing data there was in each study and how it was considered in the analysis.

The team discovered that these clinical trials routinely did not follow best practice. Most of the studies compared their treatment groups using a statistical test that can only measure whether there is a difference, not how big it is. Almost half of the studies did not estimate how differently the two treatments had performed or how much uncertainty there was around these estimates. Many studies did not explain how complete their data was, and very few looked into how any missing data affected the final results.

"Far from being a niche statistical issue, our work demonstrates a striking and worrying failure in the statistical analysis and reporting of a large number of clinical trials," explained Professor Jonathan Cook, who initiated the study. "This failure to analyse the main outcome of interest appropriately and to present the main finding of the study in an accessible way undermines the value of the study and will lead to avoidable misinterpretation – and could lead to unnecessary patient harm."

The research team hopes to use their work to raise awareness of the limitations of current practice and the need to interpret results cautiously. They also hope to contribute towards methodology and reporting guidelines to improve the quality of future binary-outcome clinical trials.

"Current practice needs to improve," concluded Prof Cook, who believes that treatment effects should be reported clearly in all papers. "Perhaps many think it is ok to leave information out of their reports as the treatment effect is 'obvious', but this leaves the reader with work to do. This isn't acceptable to me, and we as researchers have a responsibility to up our game."

Similar stories

Adalimumab is found to be a cost-effective treatment for early-stage Dupuytren’s disease

Researchers at the Kennedy Institute of Rheumatology and Oxford Population Health’s Health Economics Research Centre have found that anti-TNF treatment (adalimumab) is likely to be a cost-effective treatment for people affected by early-stage Dupuytren’s disease.

Patients like me

What can patients learn from the experiences of people like them who’ve already had a hip replacement? A new tool called ‘Patients like me’ helps answer some of the questions about pain, complications and how long the prosthesis might last.

Study reveals new evidence on rare blood-clotting condition after covid-19 vaccination

Researchers from NDORMS at the University of Oxford have investigated claims that some adenovirus-based COVID-19 vaccines increase the risk of rare blood clots compared to their mRNA-based counterparts.

NDORMS contributes to new UK research to tackle monkeypox outbreak

The UK’s efforts to tackle the monkeypox outbreak will receive a huge boost with the creation of a new research consortium and a new study to study vaccine effectiveness.

World Osteoporosis Day 2022

A bone-healthy lifestyle is vital for strong bones and a mobile, fracture-free future. On World Osteoporosis Day, we are sharing advice from the International Osteoporosis Foundation (IOF), on putting your bone health first.

Mathematician boosts data science research at the Kennedy Institute

Welcome to Yang Luo who has joined the Kennedy Institute as the Principal Investigator of the Luo Group. Her lab is leading the investigation into how genetic variations contribute to diseases of the immune system.