Basis Pursuit Support Vector Machines

• Activity Number: 159
• Type: Invited
• Date/Time: Monday, August 4, 2003 : 2:00 PM to 3:50 PM
• Sponsor: IMS
• Author(s): Hao Helen Zhang*+
• Companies: North Carolina State University
• Keywords: support vector machines ; basis pursuit ; variable selection ; smoothing spline ; ANOVA ; classification

Abstract

In many classification problems, not all the attributes are necessarily required for making predictions, and the accuracy may be increased by decreasing the number of variables used when some inputs are irrelevant or redundant to outcome. Basis pursuit support vector machine (BPSVM) is proposed to select important covariates when simultaneously training the classification rule, and obtain a compact and accurate classifier based only on important variables. In setting of a tensor product reproducing kernel Hilbert space (RKHS), the target function is decomposed into the sum of different functional components, each represented by appropriate basis functions. BPSVM minimizes the generalized comparative Kullback-Leibler distance, regulated with the l_1 penalty on the coefficients of basis terms. The solutions of BPSVM are sparse in both samples and variables; when the parameters are chosen appropriately and the RKHS is rich enough, the BPSVM estimate approaches Bayes classification rule. Numerically, BPSVM is easily implemented by solving a constrained linear programming problem. Simulations and real examples show that BPSVM is a potential tool in a wide range of applications.