Online Program Home
My Program

Abstract Details

Activity Number: 119 - SPEED: Government and Health Policy
Type: Contributed
Date/Time: Monday, July 30, 2018 : 8:30 AM to 10:20 AM
Sponsor: Health Policy Statistics Section
Abstract #330089
Title: Absence of Evidence Is Not Evidence of Absence: a Better Parallel Trends Test
Author(s): Alyssa Bilinski* and Laura Hatfield
Companies: Harvard Graduate School of Arts and Sciences and Harvard Medical School
Keywords: difference-in-differences ; DID ; parallel trends ; non-inferiority ; equivalence

Many traditional tests of statistical assumptions -- such as the parallel trends test (PTT) in a difference-in-differences (DID) analysis and the Hosmer-Lemeshow test for goodness of fit in a logistic model -- employ a null hypothesis that the assumption is met. We argue that such tests should be reformulated in a non-inferiority framework, in which the null hypothesis is that violation of the assumption exceeds a chosen threshold and the alternative is that any violation falls below that threshold ("non-inferiority margin"). We demonstrate this with the PTT in DID analysis. We show that the conventional test fails to detect many violations of parallel trends, which may be in conservative or anti-conservative directions relative to the treatment effect of interest. We propose an alternative non-inferiority approach, in which the acceptable non-inferiority margin is chosen based on the treatment effect for which the analysis is powered. We use simulation and resampling experiments to assess the performance of the proposed test in a range of scenarios and to a identify functional form that is robust to different types of violations.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2018 program