Assessing the Impact of Using County Versus State Level Outcomes Data in the Evaluation of State-Level Opioid Policies

• Activity Number: 18 - New Models, Diagnostics, and Considerations in Evaluating Intervention and Policy Effects
• Type: Topic Contributed
• Date/Time: Sunday, August 7, 2022 : 2:00 PM to 3:50 PM
• Sponsor: Health Policy Statistics Section
• Author(s): Beth Ann Ann Griffin* and Elizabeth Stuart and Megan Schuler and Nabarun Dasgupta and Rosalie Liccardo Pacula and David Powell
• Companies: RAND Corporation and Johns Hopkins University and RAND and Gillings School of Global Public Health and University of Southern California, Los Angeles, CA 90089 and RAND
• Keywords: Policy evaluations; difference-in-differences; autoregressive models; repeated measures data; panel data; observational data

Abstract

When implementing policy evaluations, it is often the case that researchers have more granular data than the policy of interest. For example, we may have county-level measures of our target outcomes (such as opioid prescribing rates) and have interest in understanding how state-level policy implementation might impact those outcomes. Researchers in these situations are often faced with the decision of whether to aggregate their rich county level data to the level at which the policy variation exists (here the state level). Yet, there is little guidance about the tradeoffs between these two approaches. We utilized Monte Carlo simulations to assess the relative performance of two different models (two-way fixed effects and autoregressive models) when utilizing county versus state-level outcomes data. The goal was to understand (i) the optimal level for modeling state-level policy impacts when both state and county level versions of the outcome are available as well (ii) the optimal specification for the different classes of models when estimating state-level policy effects on outcomes at different geographic levels.