We assess The National Weather Service's temperature forecast accuracy across 113 US cities between 2014-2017. To investigate the effect of prediction location, time of year and forecast window amongst others, we fit a multivariate normal mixed-effects model to the data. A Shiny app demonstrating our findings is shown.
Models of increasing complexity are compared. Random intercepts are added to capture the city-city variability, and a thin plate spline is used to capture spatial correlation. An unstructured variance covariance matrix is fit to capture correlations between the various forecasts made at each time point. Finally, an autoregressive time series is fit to allow for temporal correlation in the predictions.
Using our model we visually assess how the forecast accuracy and forecast bias evolve through time and space. In addition, we explore the changing magnitudes of the prediction errors as a function of forecast window. Extreme temperature events are also investigated. Finally, we test our model on unseen data to ask the question on everybody's lips: "Do I really need a jacket?"