An Empirical Assessment of the Sensitivity and Robustness of Propensity Score Estimation to Unobserved Covariates

• Activity Number: 609
• Type: Contributed
• Date/Time: Thursday, August 7, 2014 : 8:30 AM to 10:20 AM
• Sponsor: Health Policy Statistics Section
• Author(s): Wei Pan*+
• Companies: Duke University
• Keywords: propensity score analysis ; propensity score estimation ; sensitivity ; robustness ; unobserved covariates ; Pearson distributions

Abstract

As a critical component of propensity score analysis to reduce selection bias, propensity score estimation can only account for observed covariates, and behaviors of the sensitivity and robustness of propensity score estimation to unobserved covariates have not been fully understood. The present study introduces an empirical assessment of the sensitivity and robustness. The sensitivity is defined as a change from a propensity score that is estimated from a propensity score model including all observed covariates to a potential propensity score that would be estimated from the propensity score model adding an unobserved covariate. The robustness is defined as the probability of the potential change would cross a pre-specified threshold. To assess the robustness, a reference distribution of the robustness is derived by borrowing information from observed covariates and further approximated by utilizing Pearson distributions. This technique of empirical assessment of the sensitivity and robustness is illustrated with real data on substance abuse prevention for high-risk youth. The implications and limitations of the technique are also discussed.