Online Program

Evaluating the overlap weighting method for balancing covariates using real and simulated claims data (309771)

Joshua Bolton, University of Maryland 
Marshall Chin, The University of Chicago 
Elbert Huang, The University of Chicago 
Tamara Konetzka, The University of Chicago 
Manu Murugesan, The University of Chicago 
Robert Nocon, Kaiser Permanente Bernard J. Tyson School of Medicine 
*Wen Wan, The University of Chicago 

Keywords: Overlap weighting (OW); fine stratification (FS); coarsened exact matching (CEM); covariate balance; plasmode simulation; propensity score (PS); average treatment effect (ATE)

Balancing covariates is crucial for estimating causal effects in an observational study. A new method, the propensity-score (PS) based overlap weighting method (OW), produces excellent covariate balance and optimizes precision of the estimated association between exposure and outcomes. It can be used to estimate average treatment effect in the total population (ATE). The OW method has not yet been examined in a claim-based epidemiology study that quite often involves many covariates. In addition, it has only been evaluated with other weighting methods such as the inverse propensity weight method. To our knowledge, no studies has yet compared the OW method with the other types of balancing methods such as the PS-based fine stratification method (FS) and the coarsened exact matching method (CEM), both of which are popular and have been evaluated using insurance claims data. We used the Texas State Medicaid claims data on adult beneficiaries with diabetes in 2012 as an empirical example (N = 42,628) to balance 20 covariates between the federally qualified health centers (FQHC) and non-FQHC and to compare the three methods. We also used the plasmode approach to simulate outcomes and at the same time to preserve associations between covariates and exposure. The weighted generalized estimating equation models were used to estimate the FQHC effect on the simulated outcome. In the empirical example, the OW method had the smallest standardized mean differences in all covariates and Mahalanobis balance distance (MB) among the methods. In simulations, the OW method achieved the smallest MB (all scenarios <0.07), bias (<0.005), variance of bias, square root of mean squared error, and the largest coverage of the true effect size (>99.8%) than FS and CEM. These findings suggest that the OW method can yield perfect covariates balance and therefore enhance the accuracy of ATE estimation. It can be considered as a regular practical method for balancing covariates.