Sparse Forward Selection for Support Vector Classification

• Activity Number: 338
• Type: Contributed
• Date/Time: Tuesday, July 31, 2012 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Subhashis Ghoshal*+ and Hao Helen Zhang and Wookyeon Hwang
• Companies: North Carolina State University and North Carolina State University and LG Electronics
• Keywords: Classification ; forward selection ; high dimension ; support vector machines

Abstract

We propose a new binary classification technique for high dimensional predictors. Usually, only a small fraction of predictors have significant impacts on prediction. By adding an L1-type penalty in the loss function, common classification methods such as support vector machines (SVM) can perform variable selection also. We propose a method which can reduce high dimensional optimization problem to one dimensional optimization by iterating the selection step. The proposed method is based on a forward selection version of penalized SVM or its variants. The advantage of optimizing in one dimension is that the location of the optimum solution can be obtained by intelligent search by exploiting convexity and piecewise polynomial structure of the criterion function. In each step, the predictor minimizng the penalized one-dimensional loss function is chosen in the model and the step is iterated until convergence. Simulations show that the new classification rule often has better predictive and selection accuracy.