Feature Selection via Approximated Markov Blankets Using the CFS Method

Abstract

Feature selection has become an important research area in machine learning due to rapid advances in technology. In high-dimensional spaces, the difficulty of classification is intrinsically caused by the existence of irrelevant and redundant features that, in general, degrade the performance of a classifier. Moreover, finding the optimal subset of features becomes intractable even for low-dimensional datasets. In this context, Markov blanket discovery can be used to identify such subset. The approximated Markov blanket (AMb) is an efficient and effective approach to induce Markov blankets from data. However, this approach only considers pairwise comparisons of features. In this paper, we redefine the AMb to consider the interaction among features of a given subset of features. We use the Correlation based Feature Selection (CFS) function to measure such interactions and, as search strategy, the Fast Correlation based Filter (FCBF). The proposal, denoted as FCBFCFS, is compared with the FCBF and tested on synthetic and real-world datasets from the microarray domain. Results show that the inclusion of interactions among features in a subset may led to smaller subsets of features without degrading the classification task.