Pacific AB
The state of the statistical science in opioid policy research (306608)
Beth Ann Griffin, RAND CorporationSara Heins, RAND Corporation
Jill Henderson, RAND Corporation
Rosalie Pacula, RAND Corporation
Stephen Patrick, Vanderbilt University School of Medicine
David Powell, RAND Corporation
*Megan S Schuler, RAND Corporation
Rosanna Smart, RAND Corporation
Sierra Smucker, RAND Corporation
Bradley Stein, RAND Corporation
Elizabeth Stuart, Johns Hopkins Bloomberg School of Public Health
Keywords: opioid policy, statistical methods, simulation
Opioid policy research primarily uses non-experimental studies to estimate policy impacts. Our first objective was to characterize the opioid policy “state of the science” by reviewing the statistical methods used in existing opioid policy studies. To this end, we reviewed evaluation studies of state-level or federal-level opioid policies enacted in the United States published in 2005-2018. Additionally, using simulations, we compared the statistical properties of commonly used statistical methods in order to empirically identify optimal methods for specific opioid-related outcomes. Our results highlight that: (1) use of a cluster adjustment by state is critical to ensure that models have reasonable type I error rates, (2) many methods were significantly under-powered to detect a small, but meaningful and actionable, policy effect and (3) correct specification of the length of time needed until a policy becomes fully effective is vital and not fully considered in many published studies. These findings can both provide insight regarding interpretation of evidence from existing studies and as well as guide the design of future robust evaluation studies.