Algorithms for direct L2 support vector machines

Abstract

Paper introduces a novel Direct L2 Support Vector Machine (DL2 SVM) and compares the performances of its three learning algorithms on 12 `small' and 4 `medium' real binary and multi-class datasets. The DL2 SVM model is posed as solving a NonNegative (NN) Least Squares (LS) problem. This leads to a solution in much less CPU time than what the SVMs based on quadratic programming (QP) problem need. Three techniques for solving DL2 SVM's problem are the NNLS using Cholesky decomposition with an update, NN Conjugate Gradient method and a new NN Iterative Single Data Algorithm (ISDA). All 3 methods produce both high and similar classification accuracy within the very strict nested crossvalidation (a.k.a. double resampling) experimental environment, but they do significantly differ in terms of speed. Paper presents the performances of three different algorithms in terms of accuracy, model size (percentage of support vectors obtained) and CPU time used.