A Sequential Procedure for Aggregation of Expert Judgment Forecasts

• Activity Number: 339
• Type: Contributed
• Date/Time: Tuesday, July 31, 2012 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistics in Defense and National Security
• Author(s): John Irvine*+ and Srinivasamurthy R. Prakash and John Regan and Drazen Prelec
• Companies: Draper Laboratory and Draper Laboratory and Draper Laboratory and MIT
• Keywords: Forecasting ; expert judgments ; aggregation

Abstract

Policy makers rely on forecasts from experts to inform the decision making process. Merging or aggregating the judgments from multiple forecasters poses methodological challenges. New research is exploring ways of combining judgments from multiple experts to arrive at a better overall decision. For forecasts involving a small set of possible categorical outcomes where expert judgments accumulate over time, we propose a sequential procedure that chooses the best single forecast as soon as the expert judgments indicate sufficient evidence for the specific outcome. We present the formulation of this approach for binary forecasting problems. Using forecasting data for a set of real world events, we demonstrate and evaluate this new method. Comparing the performance of the proposed method to the standard unweighted linear average from the pool of subjects demonstrates the benefits of this app