Online Program Home
My Program

Abstract Details

Activity Number: 645 - Recent Developments in Score Matching with Big-Data Applications
Type: Invited
Date/Time: Thursday, August 2, 2018 : 10:30 AM to 12:20 PM
Sponsor: IMS
Abstract #326745 Presentation
Title: Scoring Rules for Probabilistic Binary Classification
Author(s): Matthew Parry*
Companies: University of Otago
Keywords: score matching; prediction; forecast; calibration

Prediction and classification challenges are an exciting feature of the statistical and machine learning community. In a probabilistic binary classification challenge, competitors are asked for a vector of marginal probabilities where each element gives the probability of assigning the corresponding case to class 1. However, in order for such a competition to be considered worth entering, the challenge organizers must be seen to evaluate the submissions in a fair and open manner. We discuss a class of proper scoring rules called linear scoring rules that are specifically adapted to probabilistic binary classification. When applied in competition situations, we show that all linear scoring rules essentially balance the needs of organizers and competitors. Furthermore, since scoring rules have a statistical decision theoretic foundation, a linear scoring rule can be constructed for any user-defined misclassification loss function. Given the scoring rule that will be used on the test cases, we also discuss the questions of training and calibrating the classifier.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2018 program