Online Program Home
  My Program

All Times EDT

Abstract Details

Activity Number: 456 - P-Values and "Statistical Significance": Deconstructing the Arguments
Type: Topic Contributed
Date/Time: Thursday, August 6, 2020 : 10:00 AM to 11:50 AM
Sponsor: Section on Statistical Consulting
Abstract #309634
Title: Rejecting Statistical Significance Tests: Defanging the Arguments
Author(s): Deborah Mayo* and Karen Kafadar* and Ya’acov Ritov* and Stanley Young*
Companies: Virginia Tech and University of Virginia and University of Michigan and CGStat
Keywords: P-values; statistical significance tests; replication crisis; strawperson fallacy; Fisher, Neyman and Pearson; 2016 ASA Statement on P-values

I critically analyze three groups of arguments for rejecting statistical significance tests (don’t say ‘significance’, don’t use P-value thresholds), as espoused in the 2019 Editorial of The American Statistician (Wasserstein, Schirm and Lazar 2019). The strongest argument supposes that banning P-value thresholds would diminish P-hacking and data dredging. I argue that it is the opposite. In a world without thresholds, it would be harder to hold accountable those who fail to meet a predesignated threshold by dint of data dredging. Forgoing predesignated thresholds obstructs error control. If an account cannot say about any outcomes that they will not count as evidence for a claim—if all thresholds are abandoned—then there is no a test of that claim. Giving up on tests means forgoing statistical falsification. The second group of arguments constitutes a series of strawperson fallacies in which statistical significance tests are too readily identified with classic abuses of tests. The logical principle of charity is violated. The third group rests on implicit arguments. The first in this group presupposes, without argument, a different philosophy of statistics from the one underlying statistical significance tests; the second group—appeals to popularity and fear—only exacerbate the ‘perverse’ incentives underlying today’s replication crisis.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2020 program