Online Program Home
  My Program

All Times EDT

Abstract Details

Activity Number: 498 - Modern Machine Learning
Type: Contributed
Date/Time: Thursday, August 6, 2020 : 10:00 AM to 2:00 PM
Sponsor: Section on Statistical Learning and Data Science
Abstract #312243
Title: Sequential Changepoint Detection for Classifier Label Shift
Author(s): Ciaran Evans* and Max G'Sell
Companies: Carnegie Mellon University and Carnegie Mellon University
Keywords: changepoint detection; classification; label shift; sequential analysis; density estimation

When applying a classifier to new data, shifts in the overall base rate of labels in the population -- known as label shift -- can lead to miscalibration of the classifier scores and significant decreases in performance. Furthermore, a shift in the overall base rate is often of practical interest in its own right. In this work, we develop a nonparametric sequential changepoint detection procedure for rapidly detecting such shifts in the population base rate based on classifier scores, without requiring observation labels. We provide methods for approximating the operating characteristics of our proposal, and show that performance can be comparable to the optimal detection procedure. In the course of this theoretical development, we also develop more general results for nonparametric changepoint detection.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2020 program