Soft partitions lead to better learned ensembles

Abstract

Ensembles of classifiers often provide better classification accuracy than a single classifier. One approach to creating ensembles is to create different subsets of the training data. We present a method of creating ensembles of classifiers by partitioning the dataset into regions using clustering. Learners are assigned to each region and the ensemble classification occurs by querying the learned classifier. The first strategy considered for partitioning the training set is to generate a hard, non-overlapping partition. This approach is shown to perform worse than a single classifier using the entire training set. However, the use of soft partitions significantly improves the overall ensemble performance. Three different methods of creating soft partitions are considered: a simple distance ratio, and both the fuzzy c-means and possibilistic c-means membership functions. All three methods are found to improve overall classifier performance beyond hard partitioning and often perform better than the base classifier using the entire training set. Experiments on six datasets illustrate the improved accuracy from creating ensembles on soft partitions of data.