Bayesian Hierarchical Stacking: All Models Are Wrong, but Some Are Somewhat Useful

• Activity Number: 308 - Highlights in Bayesian Analysis: Innovations in Bayesian Learning
• Type: Invited
• Date/Time: Tuesday, August 9, 2022 : 2:00 PM to 3:50 PM
• Sponsor: Section on Bayesian Statistical Science
• Author(s): Yuling Yao* and Gregor Pirš and Aki Vehtari and Andrew Gelman
• Companies: Flatiron Institute and University of Ljubljana and Aalto University and Columbia University
• Keywords: Bayesian hierarchical modeling; conditional prediction; covariate shift; model averaging; stacking; prior construction

Abstract

Stacking is a widely used model averaging technique. Like many other ensemble methods, stacking is more effective when model predictive performance is heterogeneous in inputs, in which case we can further improve the stacked mixture with a hierarchical model. In this talk I will focus on the recent development of Bayesian hierarchical stacking: an approach that locally aggregates models. The weight is a function of data, partially-pooled, inferred using Bayesian inference, and can further incorporate other structured priors and complex data. More generally, the success of hierarchical stacking showcases the benefit of bringing a full Bayesian workflow into an otherwise black box algorithm.