Online Program Home
My Program

Abstract Details

Activity Number: 87 - Invited ePoster Session: a Statistical Smörgåsbord
Type: Invited
Date/Time: Sunday, July 29, 2018 : 8:30 PM to 10:30 PM
Sponsor: Social Statistics Section
Abstract #329508
Title: The Consequences of Requiring 'Greater Statistical Stringency' for Scientific Publication
Author(s): Harlan Campbell* and Paul Gustafson
Companies: University of British Columbia and University of British Columbia
Keywords: meta-research; reproducibility; null hypothesis hypothesis testing; power; publication bias

In response to growing concern about the reliability and reproducibility of published science, researchers have proposed adopting measures of "greater statistical stringency", including suggestions to require larger sample sizes and to lower the highly criticized "p < 0.05" significance threshold. While pros and cons are vigorously debated, there has been little to no modelling of how adopting these measures might affect what type of science is published. In this paper, we develop a novel optimality model that, given current incentives to publish, predicts a researcher's most rational use of resources in terms of the number of studies to undertake, the statistical power to devote to each of study, and the desirable pre-study odds to pursue. We then develop a methodology that allows one to estimate the reliability of published research by sampling from a distribution of preferred research strategies. Using this approach, we investigate the merits of adopting measures of "greater statistical stringency" with the goal of informing the ongoing debate.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2018 program