Randomization Inference for Treatment Effect Variation

• Activity Number: 484
• Type: Topic Contributed
• Date/Time: Wednesday, August 12, 2015 : 8:30 AM to 10:20 AM
• Sponsor: Government Statistics Section
• Author(s): Peng Ding*
• Keywords: causal inference ; randomization test ; Head Start ; heterogeneous treatment effect

Abstract

Applied researchers are increasingly interested in whether and how treatment effects vary in randomized evaluations, especially variation not explained by observed covariates. We propose a model-free approach for testing for the presence of such unexplained variation. To use this randomization-based approach, we must address the fact that the average treatment effect, generally the object of interest in randomized experiments, actually acts as a nuisance parameter in this setting. We explore potential solutions and advocate for a method that guarantees valid tests in finite samples despite this nuisance. We also show how this method readily extends to testing for heterogeneity beyond a given model, which can be useful for assessing the sufficiency of a given scientific theory. We finally apply our method to the National Head Start Impact Study, a large-scale randomized evaluation of a Federal preschool program, finding that there is indeed significant unexplained treatment effect variation.