Twigs classifiers based on the boundary vectors Machine (BVM): A novel approach for supervised learning

Abstract

In this research, a new supervised, non-parametric and adaptive classifier is proposed: the twigs classifier. The twigs classifier uses twigs that are nothing but the boundary vectors (BVs) and their corresponding twin support vectors (SVs) found by a novel, simple and intuitive algorithm: the boundary vector bisection-based algorithm (BVB). The BVB algorithm pushes iteratively a population of scattered seeds to converge toward the boundaries between classes independently to the classes number and to the data dimensionality. Some limitations of the BVB algorithm were presented and treated. A modified version of the BVB algorithm, the BVB circle-based algorithm (BVBC), is proposed to solve the like spiral problems. The twigs classifier uses a simple dot product between the nearest twig to the object to be classified and the twig-object vector. The adaptation of the twig’s orientation and the pruning of twigs have significantly improved the classification accuracy (CA). The BVB/BVBC algorithm and the twigs classifier are evaluated and validated using synthetic and 20 UCI datasets. The efficiency of the twigs classifier is shown for multi-classification problems, for imbalanced and high-dimension data. The twigs classifier outperforms the majority of the compared classifiers among them the deep learning classifier in some cases, and achieves a misclassification rate of less than 1% in most cases.