Prognostic Score-Based Difference-in-Differences Strategy

• Activity Number: 103
• Type: Invited
• Date/Time: Monday, July 30, 2012 : 8:30 AM to 10:20 AM
• Sponsor: Social Statistics Section
• Author(s): Guanglei Hong*+ and Bing Yu and Takako Nomi
• Companies: The University of Chicago and University of Toronto and Consortium on Chicago School Research
• Keywords: Causal inference ; econometrics ; longitudinal data ; policy analysis ; propensity score ; quasi-experimental design

Abstract

In policy evaluations, the standard difference-in-differences (DID) method relies on the strong assumption that the average confounding effect of concurrent events is the same for the comparison group unaffected by the policy and the experimental group affected by the policy. Recent advancements include using propensity score matching or weighting to equate the covariate distribution between the comparison group and the experimental group. Another approach is to estimate the distribution of the counterfactual outcome of the experimental group resembling the outcome change in the comparison group. We propose an alternative strategy that involves a pair of prognostic scores per unit representing the predicted pre-policy outcome and the predicted post-policy outcome under the comparison condition in the absence of policy change. Subsequent DID analyses within subclasses defined by this pair of prognostic scores allow for a calibrated adjustment. This study compares the identification assumptions required by the prognostic score-based strategy with those of the existing strategies. We illustrate with an evaluation of a policy requiring all ninth graders to take algebra.