Online Program Home
  My Program

All Times EDT

Abstract Details

Activity Number: 75 - Contributed Poster Presentations: Biometrics Section
Type: Contributed
Date/Time: Monday, August 3, 2020 : 10:00 AM to 2:00 PM
Sponsor: Biometrics Section
Abstract #313009
Title: Effect of Different Error Rate Adjustments on Reproducibility
Author(s): Scott Richter* and Melinda McCann
Companies: UNC Greensboro and Oklahoma State University
Keywords: reproducibility; false discovery rate; multiple testing
Abstract:

Recently the science community has suffered from a “reproducibility crisis”. While there are many contributing factors, e.g., publication bias, poorly conducted experiments, another factor could be the error rate control method used. False discovery rate (FDR) is attractive for large experiments when adjusting for multiplicity. While FDR and related methods are valuable statistical additions to the science community, FDR was never intended for final confirmatory analyses. Rather FDR was designed to allow false signals in order to increase the chances that most of the true signals were identified. Unfortunately, this is not the common practice. Instead of using FDR only for initial screening to identify as many “leads” as possible, FDR is used for confirmatory experiments and the results published as rigorously establishing significance. In such cases, it is not surprising that many of the “confirmed” signals are subsequently discovered to to be false. Simulation studies investigate the reproducibility of experiments conducted using FDR and and compare those results to other methods of multiplicity control. A three-tier significance assignment is suggested to mitigate the issues.


Authors who are presenting talks have a * after their name.

Back to the full JSM 2020 program