Peer review of prediction model studies in oncology needs improvement: A systematic review of open peer review reports from BMC journals.

OBJECTIVES: To evaluate the completeness and quality of open peer review reports from BioMed Central (BMC) journals for regression-based clinical prediction model studies in oncology, focusing on adherence to methodological standards, reporting guidelines, and constructive feedback. METHODS: We searched for published prediction model studies in the field of oncology, which were published in BioMed Central journals in 2021. Data extraction used the Assessment of review Reports with a Checklist Available to eDItors and Authors (ARCADIA) checklist (13-item tool assessing review quality) with additional criteria (eg, word count, focus of comments on manuscript sections). Two investigators independently evaluated all open peer reviews, with conflicts resolved involving a third researcher. Descriptive statistics and narrative synthesis were applied. RESULTS: Peer reviews were brief (median: 243 words; range: 0-677), with 82.7% focusing on methods or results but rarely addressing limitations (<20%) or generalizability. No reviewers verified adherence to reporting guidelines (eg, TRIPOD); only one reviewer mentioned guideline use. Reviews prioritized superficial issues (67.3% focused on presentation) over methodological rigor (38.5% evaluated statistical methods). There are 19.2% suggested statistical revisions and <1% addressed protocol deviations or data availability. CONCLUSION: Our findings show that peer reviews of prediction models lack depth, methodological scrutiny, and enforcement of reporting standards. This risks clinical harm from biased models and perpetuates research waste. Reforms are urgently needed, including implementing reporting guidelines (eg, TRIPOD+AI), mandatory reviewer training, and recognition of peer review as scholarly labor. Journals must prioritize methodological rigor in reviews to ensure reliable prediction models and safeguard patient care.