Sensitivity Analysis for Grouped Data: Bias Amplification 'Danger Zones'

• Activity Number: 181
• Type: Contributed
• Date/Time: Monday, August 4, 2014 : 10:30 AM to 12:20 PM
• Sponsor: Social Statistics Section
• Author(s): Marc Scott*+ and Jennifer Hill and Ronli Diakow and Joel Middleton
• Companies: New York University and New York University and New York University and New York University
• Keywords: causal inference ; bias amplification ; sensitivity analysis ; fixed effects ; grouped data

Abstract

We are concerned with the unbiased estimation of a treatment effect in the context of observational studies with grouped data. When analyzing such data, researchers typically include as many predictors as possible, in an attempt to satisfy ignorability, and so-called fixed effects (indicators for groups) to capture unobserved between-group variation. However, adding such predictors can actually amplify treatment effect bias if ignorability is not satisfied depending on the mathematical properties of the data generating process.

We document the extent to which fixed effects act as bias amplifiers or bias reducers by comparing results from OLS regressions to fixed effects regressions in the presence of an unobserved confounder. Our approach relies on a parametric model for grouped data and an omitted confounder, establishing a framework for sensitivity analysis. We characterize the strength of the confounding along with bias amplification using easily interpretable parameters and graphical displays. We explore the mathematical conditions governing bias amplification and the extent to which one can deduce (or bound) the magnitude and direction of bias amplification from observables.