Two-Dimensional Solution Surface for Weighted Support Vector Machines

• Activity Number: 114
• Type: Topic Contributed
• Date/Time: Monday, July 30, 2012 : 8:30 AM to 10:20 AM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Seung Jun Shin*+ and Yichao Wu and Hao Helen Zhang
• Companies: North Carolina State University and North Carolina State University and North Carolina State University
• Keywords: binary classification ; probability estimation ; solution surface ; support vector machine ; weighted support vector machine

Abstract

The support vector machine (SVM) is a popular method for binary classification and its natural extension is the weighted SVM (WSVM) in which observations from different classes are imposed with different weights. There are two parameters associated with the WSVM, the regularization parameter and the weight parameter. Unlike the marginal behavior of the WSVM solution, it is largely unknown how the solution changes with respect to the two parameters jointly. We establish the joint piecewise-linearity of the WSVM solution in terms of the regularization and the weight parameter. By taking advantage of the joint piecewise-linearity, an efficient algorithm is then proposed to obtain the entire two-dimensional solution surface. Our contribution is two-fold. First we provide a theoretical understanding on how the WSVM solution changes when the two parameters vary. Second this can address assorted potential issues in practical applications as the WSVM solution is readily available for any pair of the two parameters. We demonstrate its usefulness by applying the proposed algorithm to tune the regularization parameter for the probability estimation method developed by Wang et al. (2008)