Data Augmentation for Support Vector Machines

• Activity Number: 479
• Type: Invited
• Date/Time: Wednesday, August 1, 2012 : 10:30 AM to 12:20 PM
• Sponsor: International Society of Bayesian Analysis
• Author(s): Steven Lee Scott*+ and Nicholas G Polson
• Companies: Google and The University of Chicago Booth School of Business
• Keywords: Markov chain Monte Carlo ; Bayesian inference ; Regularization ; EM algorithm ; sparse regression ; machine learning

Abstract

This talk describes a latent variable representation of regularized support vector machines. Minimizing the SVM optimality criterion, together with a parameter regularization penalty, is equivalent to finding the mode of a pseudo-posterior distribution that can be interpreted as a mean-variance mixture of normals. The latent variables in the mixture representation lead to EM and ECME point estimates of SVM parameters, as well as MCMC algorithms based on Gibbs sampling that can bring Bayesian tools for Gaussian linear models to bear on SVM's. We show how to implement SVM's with spike-and-slab priors and we illustrate the method using data from a standard spam filtering data set.