Keywords: propensity score, safety surveillance, matching, inverse probability weighting, plasmode
Safety surveillance of newly licensed drugs is challenging when there is limited uptake or low incidence of events. The FDA’s Sentinel System has developed tools that streamline the use of propensity score-based methods for assessing treatment effects. Methods that are available or under development include matching, stratification, inverse weighting, and regression modeling. We used plasmode simulations to better understand the performance of 30 variants of these methods across a set of realistic yet challenging settings. We defined 20 scenarios based on covariate associations in a real world dataset. The data generating processes varied the incidence rate, proportion treated, effect size, strength and direction of confounding, degree of overlap of treated and control populations, treatment effect heterogeneity, and presence or absence of unmeasured confounding. Study findings will help drive tool development and make it easier to choose the appropriate tool for analyzing a given dataset.