A class-specific ensemble feature selection approach for classification problems

Abstract

Due to substantial increases in data acquisition and storage, data pre-processing techniques such as feature selection have become increasingly popular in classification tasks. This research proposes a new feature selection algorithm, Class-specific Ensemble Feature Selection (CEFS), which finds class-specific subsets of features optimal to each available classification in the dataset. Each subset is then combined with a classifier to create an ensemble feature selection model which is further used to predict unseen instances. CEFS attempts to provide the diversity and base classifier disagreement sought after in effective ensemble models by providing highly useful, yet highly exclusive feature subsets. Also, the use of a wrapper method gives each subset the chance to perform optimally under the respective base classifier. Preliminary experiments implementing this innovative approach suggest potential improvements of more than 10% over existing methods.