JSM 2013 Home
Online Program Home
My Program

Abstract Details

Activity Number: 611
Type: Contributed
Date/Time: Wednesday, August 7, 2013 : 2:00 PM to 3:50 PM
Sponsor: Section on Statistical Learning and Data Mining
Abstract - #309372
Title: Computationally Efficient Confidence Intervals for Cross-Validated AUC Estimates
Author(s): Erin LeDell*+ and Maya Petersen and Mark Van der Laan
Companies: UC Berkeley and UC Berkeley - Biostatistics and UC Berkeley - Biostatistics
Keywords: Variance estimation ; Cross-validation ; Confidence intervals ; AUC ; Performance evaluation ; Bootstrap alternative

In binary classification problems, cross-validated area under the ROC curve (AUC) is a popular means of measuring the performance of your model. In order to evaluate the quality of an estimate for cross-validated AUC, we must obtain an estimate for its variance. When using ensemble learners or large data sets, the time required to generate the cross-validated AUC estimate can be significant. In these cases, the bootstrap, as a means of variance estimation, can be computationally intractable. As an alternative to the bootstrap, we demonstrate a computationally efficient influence curve based approach to obtaining a variance estimate for cross-validated AUC.

These methods for computing confidence intervals for cross-validated AUC estimates for both i.i.d. and pooled repeated measures data have been implemented in an R package called cvAUC, available on CRAN.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2013 program

2013 JSM Online Program Home

For information, contact jsm@amstat.org or phone (888) 231-3473.

If you have questions about the Continuing Education program, please contact the Education Department.

The views expressed here are those of the individual authors and not necessarily those of the JSM sponsors, their officers, or their staff.

ASA Meetings Department  •  732 North Washington Street, Alexandria, VA 22314  •  (703) 684-1221  •  meetings@amstat.org
Copyright © American Statistical Association.