An ensemble method

Abstract

This paper gives an ensemble method called EKNN-RF. Its base classifiers use an enhanced KNN algorithm where an optimal nearest neighbor number and a distance function on a validation set are obtained to make these parameters better reflect the distribution of real data. The feature set of each base classifier is obtained through bootstrap sampling from original feature set, and make the features with higher importance have a better weight. Then the training set of each base classifier is also obtained by bootstrap sampling based original training set and the newly generated feature set. Finally, each base classifier votes to determine the classification result. Experimental results show that compared with Adaboost, Naive Bayes, RandomForest, DCT-KNN [1], LMKNN+DWKNN [2], W-KNN [3], dwh-KNN [4] and LI-KNN [5], the ensemble method EKNN-RF has certain advantages and higher classification accuracy on some datasets.