Optimal Cross-Validation for Estimating Small Sample Classification Error Rate

• Activity Number: 342
• Type: Contributed
• Date/Time: Tuesday, August 8, 2006 : 2:00 PM to 3:50 PM
• Sponsor: Biometrics Section
• Author(s): Wenjiang Fu*+ and Suojin Wang
• Companies: Michigan State University and Texas A&M University
• Keywords: .632+ bootstrap ; cross-validation ; leave-one-out ; microarray data analysis ; misclassification

Abstract

Estimation of misclassification rate is important in many studies, especially in the analysis of high-throughput microarray data, where technology cost usually limits study sample size from 20 to 100. Although methods of estimating misclassification rates have been studied, many of them fail to yield satisfactory results due to small sample size, such as cross-validation, leave-one-out bootstrap (LOOBT), .632 bootstrap (BT632), etc. Although the .632+ bootstrap method (Efron and Tibshirani 1997) improves the .632 estimator by further correcting its bias, it does not perform well in terms of the overall mean squared error (MSE). In this study, we propose an optimal cross-validation method that aims to minimize the overall MSE. Results from simulation and microarray data analysis demonstrate that our novel method performs the best among its major competitors, including LOOBT and BT632.