Bayesian and Frequentist Multiplicity Control for Exclusive Hypotheses Under Dependence

• Activity Number: 155
• Type: Invited
• Date/Time: Monday, August 10, 2015 : 10:30 AM to 12:20 PM
• Sponsor: IMS
• Author(s): James Berger* and Shih-Han Chang
• Companies: Duke University and Duke University
• Keywords: multiplicity control ; exclusive testing ; prior probabilities ; frequentist properties ; dependence

Abstract

Bayesian control for multiple testing is implemented only through the choice of prior probabilities of the hypotheses. As such, it does not depend on the dependence structure of the test statistics for the various hypotheses. In contrast, many classical methods of multiplicity control such as Bonferroni, are much too conservative in the face of dependence and new methods must be developed to achieve reasonable power. The Bayesian approach will naturally yield a fully-powered procedure, but it is unclear if the approach will be acceptable to frequentists, as the frequentist properties of Bayesian multiplicity control are unknown. To study these issues we investigate, in-depth, the simplest multiple testing problem, that of testing mutually exclusive hypotheses under an exchangeable multivariate normal distribution for the test statistics. It is shown that the Bayesian procedure has remarkable power in the face of very high dependence, while achieving excellent frequentist control of multiplicity.