Research and Application of a Method for Constructing Decision Forests

Abstract

Decision forest (DF) is an implementation of multiple classifier system, which has been introduced to overcome the flaw of a single decision tree. In this paper, a method named the random subspace method (RSM) for constructing DF is investigated. RSM can keep perfect accuracy on training data while having desirable generalization accuracy. Moreover, its accuracy continues to increase as DF becomes larger, exhibiting a characteristic of overtraining resistant. The underlying theory of this model is presented in this paper. A comparison of three combination methods for this model voting, Bayesian method and neural-network is carried out. The power of voting is demonstrated both theoretically and experimentally. As for the application of this method, the superiority of RSM is explored and an advice regarding the size of attribute subsets is given.