Re-Conceptualizing the P-Value: a Probabilistic Comparison of Models Using Discrepancy Measures

• Activity Number: 31 - Methodological Advancements in Biostatistics
• Type: Contributed
• Date/Time: Sunday, July 30, 2017 : 2:00 PM to 3:50 PM
• Sponsor: ENAR
• Author(s): Benjamin Nathaniel Riedle* and Joseph Cavanaugh and Andrew Neath
• Companies: University of Iowa and University of Iowa and SIU Edwardsville
• Keywords: Discrepancy functions ; model selection ; p-value ; bootstrap

Abstract

Using observed data to choose between a null model nested within a more general alternative model is a statistical problem that arises frequently. In such settings, hypothesis testing is often employed to decide between the two models. Using this framework, the null model is favored unless the p-value is sufficiently small, in which case the null is rejected and thus the alternative is selected. In the model selection and discrepancy function framework, the model which is considered to more accurately adhere to the generating distribution is chosen. By using the bootstrap, we derive an estimate of the probability that the null model is superior to the alternative model under a specific discrepancy function. Despite some underlying differences in the hypothesis testing and discrepancy function frameworks, we show that, in certain settings, the p-value estimates the probability the null model is superior in the discrepancy function framework. Specifically, under a simple null hypothesis which pre-specifies all parameter values, we show the Wald, score and likelihood ratio test p-values are such estimates on the null.