Two-Dimensional Solution Surface for Weighted Support Vector Machines

• Activity Number: 50
• Type: Invited
• Date/Time: Sunday, August 4, 2013 : 4:00 PM to 5:50 PM
• Sponsor: Section on Statistics in Epidemiology
• Author(s): Hao Helen Zhang*+ and Seung Jun Shin and Yichao Wu
• Companies: University of Arizona and North Carolina State University and NC State University
• Keywords: svm ; solution path ; joint solution surface ; piecewise linear ; classification ; machine learning

Abstract

Support vector machine (SVM) is a popular learning method for classification. Standard SVMs treat all data points equally, but in some practical problems it is more natural to assign different weights to points from different classes. This leads to a broader class of learning called weighted SVMs (WSVMs), which has important applications in nonparametric probability estimation. There are two parameters associated with the WSVM optimization problem: the regularization parameter and the weight parameter. We establish that the WSVM solutions are jointly piecewise-linear with respect to both parameters. Motivated by this fact, we develop a state-of-the-art algorithm that computes the entire trajectory of the WSVM solutions over the two-parameter space. The two-dimensional solution surface characterizes geometric properties of the WSVM solutions, and numerically, the algorithm can greatly facilitate the WSVM implementation and automate the tuning process. We illustrate the new algorithm on various examples.