Overfitting of boosting and regularized Boosting algorithms

Abstract

The impressive generalization capacity of AdaBoost has been explained using the concept of a margin introduced in the context of support vector machines. However, this ability to generalize is limited to cases where the data does not include misclassification errors or significant amounts of noise. In addition, the research of Schapire and colleagues has served to provide theoretical support for these results from the perspective of improving margins. In this paper we propose a set of new algorithms, AdaBoost_Reg,ν-Arc, and ν-Boost, that attempt to avoid the overfitting that can occur with AdaBoost by introducing a normalization term into the objective function minimized by AdaBoost. © 2007 Wiley Periodicals, Inc. Electron Comm Jpn Pt 3, 90(9): 69– 78, 2007; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecjc.20344