A Fast Hybrid Feature Selection Method

Abstract

To confront with high dimensional datasets, several feature selection schemes have been suggested in three types of wrapper, filter, and hybrid. Hybrid feature selection methods adopt both filter and wrapper approaches by compromising between the computational complexity and efficiency. In this paper, we proposed a new hybrid feature selection method, in which in the filter stage the features are ranked according to their relevance. Instead of running the wrapper on all the features, we use a split-to-blocks technique and show that block size has a considerable impact on performance. A sequential forward selection (SFS) method was applied to the ranked blocks of features in order to find the most relevant features. The proposed method rapidly eliminates a large number of irrelevant features in its ranking stage, and then different block sizes were evaluated in the wrapper phase by choosing a proper block size using SFS. It causes this method to have a low time complexity, despite the good results. Hybrid methods consist of components that have different criteria for them. we compare and analyze different criteria. To show the effectiveness of the proposed method, state-of-the-art hybrid feature selection methods like re-Ranking, IGIS, and IGIS+ were implemented and their classification accuracies, over the known benchmarks, were computed using the K-nearest neighbor (KNN) and decision tree classifiers. Applying statistical tests to the compared results supports the superiority of the proposed method to the counterparts.