Bayes and Empirical Bayes: Will They Agree?

• Activity Number: 629
• Type: Invited
• Date/Time: Thursday, August 13, 2015 : 8:30 AM to 10:20 AM
• Sponsor: Section on Bayesian Statistical Science
• Author(s): Sonia Petrone* and Catia Scricciolo and Judith Rousseau
• Companies: Bocconi University and Bocconi University and Université Paris-Dauphine/CREST
• Keywords: Bayesian inference ; Empirical Bayes ; Merging ; Hierarchical Bayes ; Marginal maximum likelihood ; Asymptotics

Abstract

Bayesian inference is attractive due to its internal coherence and good properties. However, eliciting a honest prior can be difficult, and a common practice is to take an empirical Bayes approach using an estimate of the prior hyperparameters. Although not rigorous, the underlying idea is that, for a sufficiently large sample size, empirical Bayes methods should lead to similar inferential answers as a proper Bayesian inference. However, precise mathematical results seem to be missing. In this talk, we formalize the comparison in terms of merging of Bayesian and empirical Bayes posterior distributions. We study two notions of merging: Bayesian weak merging and frequentist merging in total variation. We also show that, under regularity conditions, the empirical Bayes approach asymptotically gives an oracle selection of the prior hyperparameters. We illustrate the results in some examples, including nonparametric density estimation and variable selection in regression. Finally, we provide evidence about the finite sample behavior of empirical Bayes and hierarchical Bayesian posterior distributions. This is a joint work with Catia Scricciolo and Judith Rousseau.