A Post-data Interpretation of Neyman-Pearson Methods Based on a Conception of Severe Testing

• Activity Number: 287
• Type: Contributed
• Date/Time: Wednesday, August 14, 2002 : 8:30 AM to 10:20 AM
• Sponsor: General Methodology
• Author(s): Deborah Mayo*+ and Aris Spanos+
• Affiliation(s): Virginia Polytechnic Institute and State University and Virginia Polytechnic Institute and State University
• Keywords: Neyman-Pearson tests ; frequentist post-data inference ; inferential interpretation ; p-values ; severity ; philosophy of statistics

Abstract

A long-standing challenge for frequentist or Neyman-Pearson (N-P) theory of testing is how it can supply a post-data assessment of evidence. The desire for a post-data measure has made the p-value appealing, but its use as a post-data measure of confirmation, critics show, leads to counterintuitive assessments. Recent attempts to solve the problem by means of "frequentist post-data" inferences have not succeeded. The root of the problem, we argue, is the unquestioned supposition that for N-P error probabilities to supply a post-data inference they would need to be interpreted as measures of confirmation or degrees of support in hypotheses. We propose a different philosophy of statistical inference in which the role of N-P error probabilities is to evaluate, post-data, the severity with which given hypotheses pass tests. Beginning with familiar N-P hypotheses tests, we will define the notion of severity and compare and contrast it to the pre-data notion of a test's power. We then sketch a general post-data reinterpretation of N-P tests based on a severity principle--that data x provide good evidence for hypotheses to the extent that the hypotheses have passed severe tests with x.