Evolving Needs of Policymakers for Drawing Inferences from Evaluation Results

• Activity Number: 628
• Type: Invited
• Date/Time: Thursday, August 13, 2015 : 8:30 AM to 10:20 AM
• Sponsor: Health Policy Statistics Section
• Author(s): Timothy James Day and Renee Mentnech and David M. Bott and Frank Yoon*
• Companies: Centers for Medicare and Medicaid Services and CMS Innovation Center and CMS Innovation Center and Mathematica Policy Research
• Keywords: policy ; policymaker ; bayesian

Abstract

Many manuscripts and reports end with phrases like, "Policymakers should consider.," but who are these policymakers and what information are they looking for in making decisions? While "policymakers" may sound like a uniform-albeit vague-group of decision makers, policymakers, in fact, represent a variety of stakeholders with varying expectations. Based on the experience of presenting research to a range of stakeholders in the federal government, we will explore the diversity of expectations stakeholders have, which directly dictate what type of information they find most useful. While statisticians and scientists tend to present findings in a binary fashion (e.g., was a result statistically significant), policymakers think in a more nuanced manner, considering how likely an inference is true and what the preponderance of (sometimes conflicting) evidence indicates. Policymakers require more flexibility than the usual statistical inference allows. Thus, we contend that using innovative ways to present inferences-such as Bayesian-derived probability statements-is crucial in allowing policymakers to weigh the evidence supporting policy actions in light of their own risk tolerances.