Robust Methods for Forecast Aggregation

• Activity Number: 490
• Type: Topic Contributed
• Date/Time: Wednesday, August 1, 2012 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Jaime Ramos*+ and David W. Scott
• Companies: Rice University and Rice University
• Keywords: Forecasting ; Aggregation ; Robust Methods ; L2E

Abstract

Adding to the well known difficulties and uncertainty involved in the forecasting process, the aggregation of hundreds or thousands of forecaster's opinions and expert predictions on social, economical and political matters makes the process even more difficult. As members of the Good Judgment Team, participating in the (IARPA) Intelligence Advanced Research Projects Activity's Aggregative Contingent Estimation (ACE) program, we are studying and developing new methods for forecast aggregation. Compared with simple quantitative data summary, regression, and maximum likelihood estimation our methods are based on L2E techniques popular in nonparametric density estimation which are robust to clusters of opinions and dramatic changes in time influenced by news, recent events, collaboration or feedback from experts. Data are analyzed under several methods and once the resolution of a particular matter is known, its accuracy is evaluated on a daily basis over the forecast period using the Brier score.