Evaluating Methods to Estimate the Effect of State Laws on Opioid-Related Outcomes in the Presence of Selection Bias

• Activity Number: 563 - Statistical Methods for State Health Policy Evaluation
• Type: Invited
• Date/Time: Thursday, August 6, 2020 : 3:00 PM to 4:50 PM
• Sponsor: Health Policy Statistics Section
• Author(s): Beth Ann Griffin*
• Companies: RAND
• Keywords: policy evaluation; difference-in-differences; selection bias; inverse probability of treatment weighting; synthetic control methods; causal inference

Abstract

The nation is in the midst of an opioid-related public health crisis. In response, states have enacted a heterogeneous collection of policies aimed at reducing mortality and morbidity, producing a state policy landscape that is complex and dynamic. Understanding how best to estimate policies’ effects is important but several unanswered questions remain, particularly about optimal methods for handling selection bias when states implementing a policy are different from states that do not. Using simulations, we examined the statistical properties (bias, power, mean square error) of several statistical methods to estimate the effects of state-level opioid policies in order to empirically identify the best methods for handling selection bias under a range of selection models (weak to strong; linear to non-linear). Findings from this study can help the field understand which methods are best to robustly estimate state-level policy effects on opioid-related outcomes. Identifying robust and powerful methods are needed to help ensure future policy decisions are based on results from well-designed evaluations that yield accurate policy effects.