Online Program

Return to main conference page
Wednesday, October 11
Wed, Oct 11, 3:30 PM - 5:00 PM
Regency Ballroom III
What Should Be the Role of Expert Opinion and Judgment in Statistical Inference and Evidence-Based Decision-Making?

What Should Be the Role of Expert Opinion and Judgment in Statistical Inference and Evidence-Based Decision-Making? (304116)

View Presentation View Presentation

*Tom Louis, The Johns Hopkins University 
*Tony O’Hagan, University of Sheffield 
*Jane Pendergast, Duke University 

The ASA’s Statement on P-Values and Statistical Significance points out the importance of not using bright-line thresholds such as p<0.05 as key factors in decision making. Strength of evidence should be viewed as continuous, not discrete. However, although decisions based on evidence should also be viewed more continuously, they often have discrete elements. (Is the drug safe and effective? Which policy intervention results in the most change? And so on.) If we are to end the practice of nearly complete reliance on a p-value below a certain threshold as a decision making filter, then we must look to other means of assessing evidence and making decisions. Overlooked when the p-value becomes the deciding factor is the role of expert opinion and judgment. In the conduct of research, expert opinion and judgment is engaged at every level. At the front end, experts decide the problem that is trying to be resolved, the study design that will help resolve it, the determination of what would be a scientifically meaningful effect and the sample size needed to measure the effect with sufficient accuracy. Experts conduct the analysis of the study data, including the determination of priors to use in Bayesian analyses. Experts interpret the results, placing them in the context of substantive knowledge in the field. Subsequently, experts review the research and determine whether it should be published. Then, finally, experts can play a continuing role in evaluating the totality of the available statistical and other evidence and making decisions or collaborating with decision makers. Assuming that no single statistical analysis provides by itself a sufficient basis for a decision, but instead provides some information about the strength of evidence in support of decision-making, what are the roles experts should play in generating, evaluating and reporting this statistical information and making decisions based on it? Are all of these potential roles for experts appropriate? How large should each role be? How is objectivity best maintained as opinion and judgment are introduced?