Online Program Home
My Program

Abstract Details

Activity Number: 116
Type: Topic Contributed
Date/Time: Monday, August 1, 2016 : 8:30 AM to 10:20 AM
Sponsor: ENAR
Abstract #320601 View Presentation
Title: On the Reproducibility of Psychology Experiments
Author(s): Valen E. Johnson*
Companies: Texas A&M University
Keywords: reproducibility ; psychology ; experiments

Investigators from a large consortium of scientists recently performed a multi-year study in which they replicated 100 psychology experiments. Although statistically significant results were reported in 97% of the original studies, only 36% of the replicated studies did. This talk examines a re-analysis of these data based on a formal statistical model that accounts for publication bias by treating outcomes from unpublished studies as missing data, while simultaneously estimating the distribution of effect sizes for those studies that tested non-null effects. The resulting model suggests that over 90% of experiments eligible for this replication study tested negligible effects, and that publication biases based on p-values subsequently caused the observed rates of non-reproducibility. The results of this re-analysis provide a compelling argument for both increasing the threshold required for declaring scientific discoveries and for adopting statistical summaries of evidence that account for the high proportion of tested hypotheses that are false.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2016 program

Copyright © American Statistical Association