A Screening Algorithm for Cross-Validated Kernel Support Vector Machines

• Activity Number: 580 - Statistical and Computational Challenges in Nonparametric Learning
• Type: Topic Contributed
• Date/Time: Thursday, August 6, 2020 : 3:00 PM to 4:50 PM
• Sponsor: Section on Nonparametric Statistics
• Author(s): Boxiang Wang* and Yi Yang
• Companies: University of Iowa and McGill University
• Keywords: Cross-validation; Reproducing kernel Hilbert spaces; Safe screening rule; support vector machines

Abstract

The kernel support vector machine (SVM) is recognized as one of the most competitive classifiers and a flagship method in data science. However, the kernel SVM has limited applications in large-scale data because of its intricate computational difficulties. In this work, we introduce a novel screening method for computing the SVM, without sacrifice of the accuracy. We strategize the screening approach to identify a subset of coefficients that are inactive in both training and leave-one-out cross-validated data, and then apply a recently proposed magic SVM algorithm to simultaneously compute the active coefficients as well as leave-one-out cross validation error. Consequently, our procedure directly yields the tuned SVM for practical use. We implement our algorithm in a publicly available R package. With simulated and real data examples, we demonstrate that our proposal is much faster than the two state-of-the-art SVM solvers: R packages kernlab and e1071.