Compact kernel classifiers trained with minimum classification error criterion

Unlike Support Vector Machine (SVM), Kernel Minimum Classification Error (KMCE) training frees kernels from training samples and jointly optimizes weights and kernel locations. Focusing on this feature of KMCE training, we propose a new method for developing compact (small scale but highly accurate) kernel classifiers by applying KMCE training to support vectors (SVs) that are selected (based on the weight vector norm) from the original SVs produced by the Multi-class SVM (MSVM). We evaluate our proposed method in four classification tasks and clearly demonstrate its effectiveness: only a 3% drop in classification accuracy (from 99.1 to 89.1%) with just 10% of the original SVs. In addition, we mathematically reveal that the value of MSVM's kernel weight indicates the geometric relation between a training sample and margin boundaries.