Online Program Home
My Program

Abstract Details

Activity Number: 617 - Testing
Type: Contributed
Date/Time: Thursday, August 1, 2019 : 8:30 AM to 10:20 AM
Sponsor: Biometrics Section
Abstract #306718 Presentation
Title: From One Environment to Many: The Problem of Replicability of Statistical Inferences
Author(s): Michael Higgins* and James J. Higgins and Jinguang Lin
Companies: Kansas State University and Kansas State University and Kansas State University
Keywords: replicability; environment by treatment interaction; treatment effect size; adjusted p-value

The environment in which an experiment is conducted is unique to each experiment. While the statistical inferences that are drawn from the analysis of experimental data apply only to the environment in which the experiment is conducted, it is almost always the intent of the researcher to apply the results more broadly. The questions then become, will statistically significant results obtained in one environment carry over to others, and if so, how much of a change in environment can be tolerated before inferences are no longer valid? We answer these questions quantitatively by proposing three measures for replicability of statistical inferences: the probability of replicability, the adjusted p-value, and the adjusted confidence interval. Through these measures, we are able to show that larger effect sizes and smaller environmental variability allow for experimental results that are more likely to be replicated. Moreover, if environmental effects are not controlled, replicating a finding may be equivalent to flipping a fair coin, regardless of how well the initial experiment was conducted.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2019 program