|
Abstract:
|
We compare the performance of three model average predictors - stacking, Bayes model averaging, and bagging - to the components that are used to form them. In all three cases we provide conditions under which the model average predictor performs as well or better than any of its components. We have limited our comparison to model averages because, even though it is well known empirically that averages provide better prediction than individual components, especially in complex problems, few theoretical results seem to be available. All three of the model averages we use can be regarded as Bayesian. Stacking is the Bayes optimal action in an asymptotic sense under several loss functions. Bayes model averaging is known to be the Bayes action under squared error, and we show that bagging can be regarded as a special case of Bayes model averaging in an asymptotic sense. We have limited our attention to the regression context since that is where model averaging techniques most conflict with current practice.
|
