A Model-Centric Approach to Scientific Progress and Understanding Reproducibility of Scientific Claims

• Activity Number: 667 - Statistics, Science, and Society
• Type: Contributed
• Date/Time: Thursday, August 2, 2018 : 10:30 AM to 12:20 PM
• Sponsor: IMS
• Author(s): Erkan Buzbas* and Berna Devezer and Bert Baumgaertner and Luis Gustavo Nardin
• Companies: University of Idaho and University of Idaho and University of Idaho and Brandenburg University of Technology Cottbus - Senftenberg
• Keywords: reproducibility; scientific process; replicability; model-centric; agent-based models; science

Abstract

Theoretical work on scientific reproducibility has hitherto assumed a hypothesis-centric view to statistical inference. Limitations of hypothesis testing, however, makes understanding salient properties of scientific process challenging, especially for fields that progress by building, comparing, selecting, and re-building models. We take a model-centric approach to scientific inquiry based on model comparisons in idealized experiments. We build a temporal stochastic process in which scientists with different research strategies search the true model generating the data. We present results on the long term probability of selecting the true model, time to hit the true model, and time to return to the true model. We investigate the effect of different research strategies, including a replicator strategy, and the complexity of the true model on these results. We show that when there is no replicator in the system, the stochastic structure of the scientific progress is simple. However, inclusion of a replicator strategy alters this stochastic structure in a complex manner. In this case, we study the properties of the scientific progress by an agent-based model.