An optimal feature selection technique using the concept of mutual information

Abstract

We present a mutual information-based technique to perform feature selection for the purpose of classification. The technique selects those features that have maximum mutual information with the specified classes. The best solution may be obtained through an exhaustive search (all possible combinations). However, even with a small number of features, this solution becomes impractical due to the exponentially increasing computational cost. Unlike other techniques that select features individually, our technique considers a trade off between computational cost and combined feature selection. Extensive experiments have shown that the proposed technique outperforms existing feature selection methods based on individual features.