Online Program

Return to main conference page
Tuesday, January 7
Tue, Jan 7, 9:00 AM - 10:45 AM
Pacific AB
Emerging Lessons on Opioid Policy Evaluation Methods

The state of the statistical science in opioid policy research (306608)

Beth Ann Griffin, RAND Corporation 
Sara Heins, RAND Corporation 
Jill Henderson, RAND Corporation 
Rosalie Pacula, RAND Corporation 
Stephen Patrick, Vanderbilt University School of Medicine 
David Powell, RAND Corporation 
*Megan S Schuler, RAND Corporation 
Rosanna Smart, RAND Corporation 
Sierra Smucker, RAND Corporation 
Bradley Stein, RAND Corporation 
Elizabeth Stuart, Johns Hopkins Bloomberg School of Public Health 

Keywords: opioid policy, statistical methods, simulation

Opioid policy research primarily uses non-experimental studies to estimate policy impacts. Our first objective was to characterize the opioid policy “state of the science” by reviewing the statistical methods used in existing opioid policy studies. To this end, we reviewed evaluation studies of state-level or federal-level opioid policies enacted in the United States published in 2005-2018. Additionally, using simulations, we compared the statistical properties of commonly used statistical methods in order to empirically identify optimal methods for specific opioid-related outcomes. Our results highlight that: (1) use of a cluster adjustment by state is critical to ensure that models have reasonable type I error rates, (2) many methods were significantly under-powered to detect a small, but meaningful and actionable, policy effect and (3) correct specification of the length of time needed until a policy becomes fully effective is vital and not fully considered in many published studies. These findings can both provide insight regarding interpretation of evidence from existing studies and as well as guide the design of future robust evaluation studies.