Cookies on this website
We use cookies to ensure that we give you the best experience on our website. If you click 'Continue' we'll assume that you are happy to receive all cookies and you won't see this message again. Click 'Find out more' for information on how to change your cookie settings.

A review of public comments on a proposed FDA regulatory framework, for modifications to artificial intelligence and machine learning-based software as a medical device, has found that 63% came from parties with financial ties to industry, and that the majority, 86% did not cite any scientific evidence.

The outside of a FDA building

The findings come from a cross-sectional study, published in BMJ Open, of the comments submitted to the US Food and Drug Administration (FDA) ‘Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD)—Discussion Paper and Request for Feedback’. 

Artificial intelligence (AI) and machine learning (ML) technologies have the potential to transform health care, continually incorporating insights from the vast amount of data generated every day during the delivery of health care. Many such devices must have regulatory approval or clearance before being available for clinical practice, and in the US that regulation falls to the FDA.

The suitability of traditional medical device regulatory pathways for AI/ML have been called into question because the nature of the technology means it is continually evolving and adapting to improve performance. Under the current framework it would mean that as devices evolved they would require further review and approval, which could be time consuming and may affect patient safety and interests. The FDA has therefore proposed a new regulatory framework for modifications to AI/ML and has asked for feedback from the public to refine the regulations.

“The process for developing regulations is, roughly, to get feedback from the public on its initial proposal, make changes and draft regulations or guidance, get more feedback, and eventually finalise,” said James Smith, Postdoctoral Scientist at the Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford and lead author of the report. “Anyone can comment but at present there is no requirement, or even recommendation, to disclose any conflicts of interest. Also, the FDA states that it looks for ‘good science’ in comments but it is not a requirement to incorporate it. Our goal was to look at the extent and disclosure of financial ties to industry and the use of scientific evidence.”

The team analysed all 125 publicly available comments on the FDA proposal between 2 April 2019 to 8 August 2019 and found that 79 (63%) comments came from parties with financial ties to industry in the sector. For a further 29% of comments the presence or absence of financial ties could not be confirmed. The vast majority of submitted comments (86%) did not cite any scientific literature, with only 4% citing a systematic review or meta-analysis.

James said: “What concerns us about these findings is that we don’t have a good idea of the impact of these ties and whether they might lead to bias in this specific context. Whether these observations about prevalence of ties hold true in the development of other regulations, we don’t yet know, but there is a growing body of evidence showing the influence of industry throughout the medical research enterprise, and this paper adds to that. I hope it will highlight the need for greater transparency.”

Gary Collins, Professor of Medical Statistics and a co-author of the study, added: “We were also concerned by the lack of scientific evidence used in comments, and the dominance of industry over academic commenters, despite AI/ML being a very active area of research. But we hope our findings will bring the FDA proposal to the attention of academics and encourage more of them to participate in the next round of feedback on the framework, and other regulatory frameworks, where academic input could be valuable.”

The study was funded by the NIHR Oxford Biomedical Research Centre (BRC).

Similar stories

NDORMS researchers awarded for Dupuytren research

Awards Hand Kennedy Main

Three NDORMS researchers have received awards from the International Dupuytren Society, a patient organisation that brings together Dupuytren Disease patient societies from across the world.

Hope for rheumatoid arthritis patients who are non-responsive to anti-TNF

Arthritis Kennedy Main

New research published in The Lancet shows that tocilizumab is a more effective treatment than rituximab for rheumatoid arthritis patients with a poor response to anti-tumour necrosis factor (TNF).

A new study maps the expression of innate immune receptors during the course of arthritis

Arthritis Kennedy Main

The research, which was a collaboration with researchers from Oxford University and Queen Mary University of London and published in Journal of Autoimmunity, looked at changes in receptors known as toll-like receptors (TLRs) in arthritis at different stages of disease.

International Women's Day

Department Main

It’s International Women's Day! This year’s theme is #Choosetochallenge. We’re celebrating some of the amazing women at NDORMS, and asking them what changes they’d like to see in medical sciences over the next 100 years.

Patients and carers invited to join new group helping to shape research and treatment of bones, muscles and joints

Main PPI

Oxford’s newest patient partner group, OPEN ARMS launches today to explore the causes, treatment and care for patients with musculoskeletal conditions. Its first three patient partners explain why they are involved and invite other members of the public to join the team.

NDORMS academics named NIHR Senior Investigators

Main

Congratulations to Professor Jonathan Rees who has been announced as a National Institute of Health Research (NIHR Senior Investigator).