|
Abstract:
|
Distance weighted discrimination (DWD) is a modern margin-based classifier with an interesting geometric motivation. Despite many recent papers on DWD, DWD is far less popular compared with the support vector machine (SVM), mainly due to computational and theoretical reasons. In this work, we greatly advance the current DWD methodology and its learning theory. We propose a novel efficient algorithm for solving DWD, and our algorithm can be several hundred times faster than the existing state-of-the-art algorithm based on the second order cone programming (SOCP). In addition, our algorithm can handle the generalized DWD, while the SOCP algorithm only works well for a special DWD but not the generalized DWD. Furthermore, we formulate a natural kernel DWD in a reproducing kernel Hilbert space and then establish the Bayes risk consistency of the kernel DWD using a universal kernel such as the Gaussian kernel. This result solves an open theoretical problem in the DWD literature. We compare DWD and the SVM on several benchmark data sets and show that the two have comparable classification accuracy, but DWD equipped with our new algorithm can be much faster to compute than the SVM.
|
