Feature Selection based on Particle Swarm Optimization and Mutual Information

Abstract

Today, feature selection, as a technique to improve the performance of the classification methods, has been widely considered by computer scientists. As the dimensions of a matrix has a huge impact on the performance of processing on it, reducing the number of features by choosing the best subset of all features, will affect the performance of the algorithms. Finding the best subset by comparing all possible subsets, even when n is small, is an intractable process, hence many researches approach to heuristic methods to find a near-optimal solutions. In this paper, we introduce a novel feature selection technique which selects the most informative features and omits the redundant or irrelevant ones. Our method is embedded in PSO (Particle Swarm Optimization). To omit the redundant or irrelevant features, it is necessary to figure out the relationship between different features. There are many correlation functions that can reveal this relationship. In our proposed method, to find this relationship, we use mutual information technique. We evaluate the performance of our method on three classification benchmarks: Glass, Vowel, and Wine. Comparing the results with four state-of-the-art methods, demonstrates its superiority over them.