Abstract:
|
Echo state networks (ESN) have been shown to be a powerful tool for the prediction of time series and spatio-temporal data. ESNs are a reservoir computing version of recurrent neural networks (RNNs) in which the weight matrices controlling hidden layers are generated randomly, thus providing a huge computational advantage over traditional RNNs. Deep versions of the ESN consist of multiple hidden layers, again using randomly generated weight matrices. The principle challenge for statistical consideration of such models is uncertainty quantification (UQ) and interpretation. Here we demonstrate for the first time that effective UQ can be accomplished by coupling the ESN to a mixture density network (MDN), and that interpretation of the fitted network can be accomplished via Layerwise Relevance Propagation (LRP). Specifically, LRP can determine which input components are most important in generating forecasts with the model. The approach is demonstrated via simulation study based on noisy versions of classic nonlinear dynamics models. Finally, we apply the approach to long-lead forecasting of Pacific sea surface temperature data.
|