Multi-Step Forecast Model Selection and Combination

• Activity Number: 248
• Type: Contributed
• Date/Time: Monday, August 1, 2011 : 2:00 PM to 3:50 PM
• Sponsor: Business and Economic Statistics Section
• Author(s): Bruce E. Hansen*+
• Companies: University of Wisconsin
• Keywords: MSFE ; cross-validation ; forecast combination ; model averaging ; information criterion ; multi-step forecasting

Abstract

This paper examines model selection and combination in the context of multi-step linear forecasting. We start by investigating multi-step mean squared forecast error (MSFE). We derive the bias of the in-sample sum of squared residuals as an estimator of the MSFE. We find that the bias is not generically a scale of the number of parameters, in contrast to the one-step-ahead forecasting case. Instead, the bias depends on the long-run variance of the forecast model. In consequence, standard information criterion (Akaike, FPE, Mallows and leave-one-out cross-validation) are biased estimators of the MSFE in multi-step forecast models. These criteria are generally under-penalizing for over-parameterization and this discrepancy is increasing in the forecast horizon. In contrast, we show that the leave-h-out cross validation criterion is an approximately unbiased estimator of the MSFE and is thus a suitable criterion for model selection. Leave-h-out is also suitable for selection of model weights for forecast combination. We provide strong simulation and empirical evidence in favor of weight selection by leave-h-out cross validation.