Model Evaluation, Discrepancy Function Estimation, and Social Choice Theory

• Activity Number: 190
• Type: Contributed
• Date/Time: Monday, July 30, 2012 : 10:30 AM to 12:20 PM
• Sponsor: Section on Risk Analysis
• Author(s): Andrew Neath*+ and Joseph Cavanaugh and Adam Weyhaupt
• Companies: SIU Edwardsville and University of Iowa and SIU Edwardsville
• Keywords: model selection ; resampling ; conceptual predictive statistic

Abstract

A discrepancy function provides for an evaluation of a candidate model by quantifying the disparity between the candidate model and the true model which generated the observed data. The best model from a candidate class is the one judged to have minimum discrepancy with the true model. Since data provides only partial information on the underlying true model, we consider taking repeated simulations using some resampling procedure as each representing some possibility for the true model. Each of the candidate models can be judged against each of the simulated versions of the true model. Model evaluation is then achieved by providing an overall judgment on each candidate model. As social choice theory, or voting theory, addresses the problem of turning individual preferences into a group preference, we see that social choice theory can be used in developing a new approach to model evaluation.