Evaluation of Wine Judge Performance Through Three Characteristics: Bias, Discrimination, and Variation

• Activity Number: 414
• Type: Topic Contributed
• Date/Time: Tuesday, August 11, 2015 : 2:00 PM to 3:50 PM
• Sponsor: Business and Economic Statistics Section
• Author(s): Jing Cao* and Lynne Stokes
• Companies: and Southern Methodist University
• Keywords: Bayesian model ; ordinal data analysis ; wine competition ; rater agreement

Abstract

Judge performance is a critical component of a wine competition's success. A number of studies have shown that wine judges may differ considerably in their opinions. In this paper, we have conducted an in-depth examination of wine judge performance at a U.S. wine competition. Three characteristics of judge's performance are examined: bias, discrimination ability, and variation. Based on the analysis, we can identify the judges who had discrepant scoring patterns and can gain insight into which of the three characteristics cause particular judges to disagree. The evaluation of wine judge performance through these three characteristics may provide useful information for training them to have consistent performance and in assisting competition coordinators in judge selection.