Assess the Strength of Statistical Evidence, Not the P-value

• Activity Number: 120
• Type: Invited
• Date/Time: Monday, August 8, 2005 : 10:30 AM to 12:20 PM
• Sponsor: Biopharmaceutical Section
• Author(s): Jeffrey D. Blume*+
• Companies: Brown University
• Keywords: Statistical evidence ; p-value ; likelihood ratio ; misleading evidence

Abstract

It is important to assess the strength of statistical evidence in the data themselves. But the p-value is, at best, a clumsy tool for making such an assessment. For example, two studies with different sample sizes for the same drug may yield the same p-value even though the strength of evidence between them is different. In this paper, I argue that likelihood ratios are a better tool for measuring the strength of statistical evidence. Some of their attractive properties are (1) the probability of observing misleading evidence and weak evidence (analogous quantities to the type I and II errors) both converge to zero as the sample size increases; (2) the probability of observing misleading evidence is low and controllable at any sample size; (3) even with multiple looks at the data, the probability of observing misleading evidence remains bounded and controllable (although it increases with each look at the data, the amount by which it increases converges to zero); (4) likelihood ratios do not depend on prior distributions or sample spaces; and (5) a simple empirical adjustment of the likelihood ratio makes it robust to model misspecification.