Support Vector Machines and Statistical Models

• Activity Number: 88
• Type: Invited
• Date/Time: Monday, August 12, 2002 : 10:30 AM to 12:20 PM
• Sponsor: SSC
• Author(s): Trevor Hastie*+ and Ji Zhu
• Affiliation(s): Stanford University and Stanford University
• Keywords: SVM ; Logistic Regression ; RKHS

Abstract

The SVM techniques pioneered by Vladimir Vapnik have created a growth industry in computer science and machine learning. Originally developed as enhancements of the separating hyperplane for two-class classification, we now have SVM versions of regression, principal components, time-series models--and the crank is still turning.

In this talk, I describe the SVM, and its use of "inner-product" kernels to achieve flexible generalizations. I then view the SVM as as the minimization of a regularized error function in a reproducing kernel Hilbert space of functions, with strong connections to the smoothing spline technology of Grace Wahba. Our conclusion is that the SVM looks very much like nonparametric logistic regression, without all the benefits, such as a multi-class generalization and the interpretation of the fitted functions as logistic transformations of class probabilities.