Variable Selection for Support Vector Machine on High Dimensions

• Activity Number: 413
• Type: Topic Contributed
• Date/Time: Tuesday, August 6, 2013 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Xiang Zhang*+ and Lan Wang and Runze Li and Yichao Wu
• Companies: North Carolina State University and University of Minnesota and The Pennsylvania State University and NC State University
• Keywords: SVM ; Variable Selection ; Oracle Property ; LLA

Abstract

Support Vector Machine (SVM) is a popular classification tool. However, it selects all variables and can perform poorly in high dimensional space due to noise accumulation. In this paper we address the variable selection problem of SVM and prove the oracle property of our procedure. We show that under weak conditions, for a general class of nonconvex penalty, one of the local minimizer of nonconvex penalized SVM is the oracle estimator, that is, we estimate the model as if the true model is known in advance. We also provide a non-asymptotic lower bound for the probability of identifying the oracle from possibly multiple local minima. Furthermore, we give sufficient conditions under which the oracle is found by local linear approximation algorithm with probability tending to one.