Binary particle swarm optimisation and rough set theory for dimension reduction in classification

Abstract

Dimension reduction plays an important role in many classification tasks. In this work, we propose a new filter dimension reduction algorithm (PSOPRSE) using binary particle swarm optimisation and probabilistic rough set theory. PSOPRSE aims to maximise a classification performance measure and minimise a newly developed measure reflecting the number of attributes. Both measures are formed by probabilistic rough set theory. PSOPRSE is compared with two existing PSO based algorithms and two traditional filter dimension reduction algorithms on six discrete datasets of varying difficulty. Five continues datasets including a large number of attributes are discretised and used to further examine the performance of PSOPRSE. Three learning algorithms, namely decision trees, nearest neighbour algorithms and naive Bayes, are used in the experiments to examine the generality of PSOPRSE. The results show that PSOPRSE can significantly decrease the number of attributes and maintain or improve the classification performance over using all attributes. In most cases, PSOPRSE outperforms the first PSO based algorithm and achieves better or much better classification performance than the second PSO based algorithm and the two traditional methods, although the number of attributes is slightly large in some cases. The results also show that PSOPRSE is general to the three different classification algorithms.