Minimizing the Misclassification Rate of the Nearest Neighbor Rule Using a Two-stage Method

Abstract

The kNN classification performance entirely depends on the selected neighbors. In the past, many nearest neighbor (NN)-based methods mainly focus on learning distance measure metrics so that a neighborhood of an approximately constant posteriori probability can be produced, whereas limited works are performed to study the influences of the distribution characteristics of each neighbor. In this paper, we point out why the best distance measurement (BDM) is sensitive to malicious samples, and then a robust best distance measurement (RBDM) is suggested to solve this problem. Moreover, we also investigated the influences of the distribution characteristics of each neighbor for the classification performance, so that a two-stage method, called weighted robust best distance measurement kNN method (WRBDMkNN), is proposed aiming to minimize the misclassification rate of the nearest neighbor rule. Extensive experiments on diversity datasets indicate that the proposed method can achieve more encouraging results compared with some state-of-the-art NN-based methods.