Some Comments on CV

• Activity Number: 400 - Breiman Award Lectures
• Type: Invited
• Date/Time: Thursday, August 12, 2021 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Learning and Data Science
• Author(s): Trevor JOHN Hastie* and Stephen Bates and Robert Tibshirani
• Companies: STANFORD UNIVERSITY and UC Berkeley and Stanford Univ
• Keywords: Cross-validation; OOB; Error Estimation; Bootstrap

Abstract

Cross-validation is ubiquitous in data science, and is used for both model selection and assessment. Yet in some regards it is poorly understood. In this talk we discuss three aspects of CV.

1) What CV estimates. 2) Confidence intervals for prediction error using nested CV. 3) OOB error for random-forests and standard error estimates.

This talk is dedicated to Leo Breiman and Colin Mallows