The performance of different strategies to manage confounding in cardinality matching for medical device epidemiology: A simulation study

Background: Residual confounding, either related to patient or surgeon variables, leads to bias in the estimation of treatment effects in observational studies of medical devices. Objectives: To estimate the performance (bias and precision) of confounder balance criteria settings for cardinality matching in medical device epidemiology studies. Methods: Cardinality matching (CM) is a matching method that finds the largest matched sample according to user’s prespecified confounder balance criteria. Multi-level Monte Carlo simulations (1,000 iterations) with patients nested under surgeon (ratio 500:1) and sample size of n=10,000 were conducted. Five patient confounders, one instrumental variable and one risk factor were generated, all binary and based on a bernoulli distribution. Fixed true treatment effect at Odds ratio (OR) 1.5. A surgeon-level confounder was generated with Pois(2), with an OR = 2 association with treatment choice, and an OR ranging from 1.01 to 5 for effect on outcome. CM was used to balance the confounders. A range of different confounder balance criteria were tested as part of CM, namely standardised mean difference (SMD) of 0, 0.001, 0.01, and 0.1. Treatment effects were then estimated using logistic regression as the outcome model on the matched sample obtained from CM. The resulting treatment effects were compared to the true effect, and % bias and root mean square errors (RMSE) estimated. Results: Confounder balance of SMD = 0.1 results in lower bias and RMSE for weaker surgeon effects on outcome (OR <= 1.5), whilst SMD < 0.1 performed best for strong surgeon confounding scenarios (OR > 1.5). For example, for an OR of 1.25 for surgeon effect on outcome, %bias and RMSE for SMD = 0.1 was 5.4% and 0.056 compared to 11.6% and 0.069 for SMD = 0.01. Conversely, for OR of 2.5 for surgeon effects, %bias and RMSE for SMD = 0.1 were 24.1% and 0.124 respectively, compared to 1.1% and 0.056 for SMD = 0.01. Conclusions: Confounder balance choice for CM impacts the bias, and should be informed by observable effects of surgeon on outcome. More research is needed to guide the use of CM in medical device epidemiology.