Cookies on this website
We use cookies to ensure that we give you the best experience on our website. If you click 'Continue' we'll assume that you are happy to receive all cookies and you won't see this message again. Click 'Find out more' for information on how to change your cookie settings.

The statistical analysis and reporting of treatment effects in reports of randomised trials with a binary primary endpoint requires substantial improvement, suggests NDORMS research published in BMC Medicine.

Evaluation checklist © Shutterstock

 

Clinical trials help doctors decide between different treatments and ensure that patients receive the best care possible. Good decision-making is only possible if these trials are done correctly and if doctors have access to all of the relevant information about these trials in their resulting academic publications.

A research team from the Oxford Clinical Trials Research Unit (OCTRU) and Centre for Statistics in Medicine, NDORMS, has found evidence that trials with binary outcomes are particularly badly reported. Around half of all clinical trials use binary outcomes. These trials look at two possible outcomes rather than a continuum, such as failure or success of a treatment, or whether patients are dead or alive at the end of the trial.

The researchers looked at 200 academic publications of clinical trials published in January 2019 that used a binary outcome as the main finding of the study. They examined the statistical methods used to analyse the data and whether the publication merely assessed if there was a difference between the treatment groups or included details of how big this difference was and how it was calculated. The researchers also measured how much missing data there was in each study and how it was considered in the analysis.

The team discovered that these clinical trials routinely did not follow best practice. Most of the studies compared their treatment groups using a statistical test that can only measure whether there is a difference, not how big it is. Almost half of the studies did not estimate how differently the two treatments had performed or how much uncertainty there was around these estimates. Many studies did not explain how complete their data was, and very few looked into how any missing data affected the final results.

"Far from being a niche statistical issue, our work demonstrates a striking and worrying failure in the statistical analysis and reporting of a large number of clinical trials," explained Professor Jonathan Cook, who initiated the study. "This failure to analyse the main outcome of interest appropriately and to present the main finding of the study in an accessible way undermines the value of the study and will lead to avoidable misinterpretation – and could lead to unnecessary patient harm."

The research team hopes to use their work to raise awareness of the limitations of current practice and the need to interpret results cautiously. They also hope to contribute towards methodology and reporting guidelines to improve the quality of future binary-outcome clinical trials.

"Current practice needs to improve," concluded Prof Cook, who believes that treatment effects should be reported clearly in all papers. "Perhaps many think it is ok to leave information out of their reports as the treatment effect is 'obvious', but this leaves the reader with work to do. This isn't acceptable to me, and we as researchers have a responsibility to up our game."

Similar stories

NDORMS researchers awarded for Dupuytren research

Awards Hand Kennedy Main

Three NDORMS researchers have received awards from the International Dupuytren Society, a patient organisation that brings together Dupuytren Disease patient societies from across the world.

Hope for rheumatoid arthritis patients who are non-responsive to anti-TNF

Arthritis Kennedy Main

New research published in The Lancet shows that tocilizumab is a more effective treatment than rituximab for rheumatoid arthritis patients with a poor response to anti-tumour necrosis factor (TNF).

A new study maps the expression of innate immune receptors during the course of arthritis

Arthritis Kennedy Main

The research, which was a collaboration with researchers from Oxford University and Queen Mary University of London and published in Journal of Autoimmunity, looked at changes in receptors known as toll-like receptors (TLRs) in arthritis at different stages of disease.

International Women's Day

Department Main

It’s International Women's Day! This year’s theme is #Choosetochallenge. We’re celebrating some of the amazing women at NDORMS, and asking them what changes they’d like to see in medical sciences over the next 100 years.

Patients and carers invited to join new group helping to shape research and treatment of bones, muscles and joints

Main PPI

Oxford’s newest patient partner group, OPEN ARMS launches today to explore the causes, treatment and care for patients with musculoskeletal conditions. Its first three patient partners explain why they are involved and invite other members of the public to join the team.

NDORMS academics named NIHR Senior Investigators

Main

Congratulations to Professor Jonathan Rees who has been announced as a National Institute of Health Research (NIHR Senior Investigator).