Kernel-free Reduced Quadratic Surface Support Vector Machine with 0-1 Loss Function and L\(_p\)-norm Regularization

Abstract

This paper presents a novel nonlinear binary classification method, namely the kernel-free reduced quadratic surface support vector machine with 0-1 loss function and L\(_{p}\)-norm regularization (L\(_p\)-RQSSVM\(_{0/1}\)). It uses kernel-free trick aimed at finding a reduced quadratic surface to separate samples, without considering the cross terms in quadratic form. This saves computational costs and provides better interpretability than methods using kernel functions. In addition, adding the 0-1 loss function and L\(_p\)-norm regularization to construct our L\(_p\)-RQSSVM\(_{0/1}\) enables sample sparsity and feature sparsity. The support vector (SV) of L\(_p\)-RQSSVM\(_{0/1}\) is defined, and it is derived that all SVs fall on the support hypersurfaces. Moreover, the optimality condition is explored theoretically, and a new iterative algorithm based on the alternating direction method of multipliers (ADMM) framework is used to solve our L\(_p\)-RQSSVM\(_{0/1}\) on the selected working set. The computational complexity and convergence of the algorithm are discussed. Furthermore, numerical experiments demonstrate that our L\(_p\)-RQSSVM\(_{0/1}\) achieves better classification accuracy, less SVs, and higher computational efficiency than other methods on most datasets. It also has feature sparsity under certain conditions.