Some notes on the basic concepts of support vector machines

Abstract

Support vector machines (SVMs) are classic binary classification algorithms and have been shown to be a robust and well-behaved technique for classification in many real-world problems. However, there are ambiguities in the basic concepts of SVMs although these ambiguities do not affect the effectiveness of SVMs. Corinna Cortes and Vladimir Vapnik, who presented SVMs in 1995, pointed out that an SVM predicts through a hyperplane with a maximal margin. However existing literatures have two different definitions of the margin. On the other hand, Corinna Cortes and Vladimir Vapnik converted an SVM into an optimization problem that is much easier to solve. Nevertheless, existing papers do not explain how the optimization problem derives from an SVM well. These ambiguities may cause certain troubles in understanding the basic concepts of SVMs. For this purpose, this paper defines a separating hyperplane of a training data set and, hence, an optimal separating hyperplane of the set. The two definitions are reasonable since this paper proves that ${\text{w}}_0^{\text{T}}\text{x}+b_0=0$ is an optimal separating hyperplane of a training data set when ${\text{w}}_0$ and $b_0$ constitute a solution to the above optimization problem. Some notes on the above margin and optimization problem are given based on the two definitions. These notes should be meaningful for clarifying the basic concepts of SVMs.