Interpreting P-Values and Confidence Intervals Using Well-Calibrated Null Preference Priors

• Activity Number: 34 - Foundations in Bayesian Statistics
• Type: Contributed
• Date/Time: Sunday, July 28, 2019 : 2:00 PM to 3:50 PM
• Sponsor: Section on Bayesian Statistical Science
• Author(s): Michael Fay* and Michael Proschan and Erica Brittain and Ram Tiwari
• Companies: National Institute of Allergy and Infectious Diseases and National Institute of Allergy and Infectious Diseases and National Institute of Allergy and Infectious Diseases and CDRH, FDA
• Keywords: p-values; Bayesian-frequentist comparison; objective priors; calibration; hypothesis testing; confidence intervals

Abstract

We propose well-calibrated null preference priors for use with one-sided hypothesis tests, such that resulting Bayesian and frequentist inferences agree. Null preference priors mean that they have essentially 100% of their prior belief in the null hypothesis, and well-calibrated priors mean that the resulting posterior beliefs in the alternative hypothesis are not overconfident. Under this framework, the posterior belief in the null hypothesis is the p-value, and the null preference prior emphasizes that large p-values may simply represent insufficient data to overturn the prior belief. This framework allows us to shed light on the negative binomial/binomial controversy, some two sample tests, and some problems with two-sided tests.