Online Program

Evolving Needs of Policymakers for Drawing Inferences from Evaluation Results

*Randall Brown, Mathematica Policy Research 

Keywords: policymakers, Bayesian, federal, government

Many manuscripts and reports end with phrases like, “Policymakers should consider…,” but who are these policymakers, and what information are they looking for in making decisions? While “policymakers” may sound like a uniform—albeit vague—group of decision-makers, policymakers, in fact, represent a variety of stakeholders with varying expectations. Based on the experience of presenting research to a range of stakeholders in the federal government, we will explore the diversity of expectations stakeholders have, which directly dictate what type of information they find most useful. While statisticians and scientists tend to present findings in a binary fashion (e.g., was a result statistically significant), policymakers think in a more nuanced manner, considering how likely an inference is true and what the preponderance of (sometimes conflicting) evidence indicates. Policymakers require more flexibility than the usual statistical inference allows. Thus, we contend that using innovative ways to present inferences—such as Bayesian-derived probability statements—is crucial in allowing policymakers to weigh the evidence supporting policy actions in light of their own risk tolerances.