A Bayes Interpretation of Stacking for M-Open Settings

• Activity Number: 354
• Type: Contributed
• Date/Time: Tuesday, August 11, 2015 : 10:30 AM to 12:20 PM
• Sponsor: Section on Bayesian Statistical Science
• Author(s): Tri Le* and Bertrand Clarke
• Companies: University of Nebraska - Lincoln and University of Nebraska - Lincoln
• Keywords: M-open ; stacking predictor ; cross-validation ; utility ; basis selection ; bootstrap

Abstract

Stacking is a model averaging method that constructs a predictor from a list of predictors using an optimization based on cross-validation. In M-open settings where no true model can be conceptualized we provide a decision-theoretic Bayes interpretation for the coefficients in the stacking predictor. In this setting, the stacking weights merely index a class of actions and cannot be interpreted as prior weights. In particular, the stacking weights need not satisfy the non-negativity or "sum to one constraint" in general. Nevertheless, we show that the weights are asymptotically equivalent to finding the Bayes action under a posterior risk.

To construct stacking predictors in practice, we suggest that the predictors being stacked should be different from each other in the sense of being independent; orthogonality does not seem to be helpful. In this context, we suggest using bootstrap samples from the data to generate empirical basis elements in a function space. Now we can find a stacking predictor that is optimal with respect to the number of component predictors as well as with respect to the weights each gets and does not require the analyst to choose which predictors to stack.