An Improved Density Based Support Vector Machine (DBSVM)

Abstract

Support Vector Machines (SVM) is a classification model based on the duality optimization approach. Non-zero Lagrange multipliers correspond to the data selected to be support vectors used to build the margin decision. Unfortunately, SVM has two major drawbacks: the noisy and redundant data cause an overfitting; moreover, the number of local minima increases with the size of data, even worse when it comes to Big Data. To overcome these shortcoming, we propose a new version of SVM, called Density Based Support Vector Machine (DBVSM), which performs on three steps: first, we set two parameters, the radius of the neighborhood and the size of this latter. Second, we determine three types of points: noisy, cord and interior. Third, we solve the dual problem based on the cord data only. To justify this choice, we demonstrate that the cord points cannot be support vectors. Moreover, we show that the kernel functions don't change the cord point nature even. The DBSVM is benchmarked on several datasets and is compared with a variety of methods in the literature. The results of the tests prove that the proposed algorithm is able to provide very competitive results in terms of time, classification performance, and capacity to tackle datasets of very large size. Finally, to point out the consistency of the DBSVM, several tests were performed for different values of the ratio and the neighborhood size.