OBJECTIVES: To systematically review and critically appraise the methodology of developing modified obstetric early warning scores (MOEWSs). STUDY DESIGN AND SETTING: We searched Medline, CINAHL, EMBASE, and the Web of Science for MOEWS studies published between January 1, 2000, and December 31, 2022. Eligible studies included models predicting maternal death, intensive care unit (ICU) admission, and/or a composite of two or more maternal morbidities occurring in a hospital setting in women of any gestational age and up to 1 week after the end of pregnancy. Models were critically appraised using the Prediction Model Risk of Bias Assessment Tool (PROBAST) and adherence to the transparent reporting of prediction models (TRIPOD). RESULTS: 20 studies were included: five (25%) were model development studies, five (25%) were model development and validation, and ten (50%) were validation only. Four development studies used statistical methods, and the remaining six studies used clinical consensus (ie, expert opinion). The four data-driven model development studies did not address key statistical challenges, such as repeated measures or missing data, nor did they assess the performance adequately or dataset characteristics clearly. All but one study (95%) were rated at high risk of bias due to data sources, poor reporting, and analysis limitations. The fifteen validation studies were poorly reported and eleven (73%) were at high risk of bias. None of the data-driven models were independently validated, a key step toward implementation. CONCLUSION: There is a lack of MOEWSs developed using methods that follow recommended statistical guidelines. Substantial problems with the methodological quality of included development and validation studies, along with high risk of bias,indicating published scores could perform poorly or be potentially harmful if used in clinical practice. Future work should address handling missing data and repeated measures and consider how an MOEWS will perform in different populations and key subgroups.