Class noise elimination approach for large datasets based on a combination of classifiers

Abstract

Noise points, or class noise, detection and elimination became increasingly important to handle large datasets. In fact, eliminating noise in this environment helps reduce computing costs, especially when using clustering algorithms. Nowadays, large varieties of clustering algorithms exist and produce good results. However, they often assume that the input data are free or have very low level of noise, which is rarely the case in real Big Data context. In this paper, we present a noise detection and elimination approach for large datasets. This approach relies on four important steps: divide data into subsets, extract the best rules, apply different classifiers to the subsets, and finally combine the classifiers results.