Causal Inference with Complex Surveys: a Comparison of Propensity Score Based Methods

• Activity Number: 340 - SPEED: Applications of Advanced Statistical Techniques in Complex Survey Data Analysis: Small Area Estimation, Propensity Scores, Multilevel Models, and More
• Type: Contributed
• Date/Time: Tuesday, July 31, 2018 : 10:30 AM to 12:20 PM
• Sponsor: Survey Research Methods Section
• Author(s): Daniele Bottigliengo* and Ileana Baldi and Corrado Lanera and Dario Gregori and Paola Berchialla
• Companies: Università degli Studi di Padova and Università degli Studi di Padova and Università degli Studi di Padova and Università degli Studi di Padova and Università degli Studi di Torino
• Keywords: Causal inference; Complex surveys; Propensity Score Matching; Propensity Score Inverse Probability Treatment Weighting; Double Robust Estimator; Monte Carlo simulations

Abstract

Introduction. Propensity Score (PS) methods are widely applied to infer the effect of a treatment from observational data. However, their use in complex survey designs, which require sampling weights, has been limited so far. The goal of this study is to review and compare some of the most popular PS approaches in the context of complex surveys, focusing on how to include sampling weights in the analysis. Methods. Comparison was carried through out Monte Carlo simulations and on real-world data. Propensity Score Matching (PSM), Propensity Score as Inverse Probability of Treatment Weighting (PS-IPTW) and Double Robust estimator (DR) were evaluated in terms of bias, Mean Squared Error (MSE) and Nominal Coverage. Bootstrap with 500 repetitions was used to estimate standard errors and confidence intervals. Results. Including sampling weights in PS-IPTW and DR at both PS and outcome level led to lower bias and MSE. PSM performed better without sampling weights. However, the three approaches showed bias and MSE below 5% and 1% respectively and Nominal Coverage higher than 90%. Conclusions. In complex survey designs, we recommend using sampling weights for both PS-IPTW and DR methods.