A systematic method to guide the choice of ridge parameter in ridge extreme learning machine

Abstract

Extreme Learning Machine (ELM) has attracted many researchers as a universal function approximator because of its extremely fast learning speed and good generalization performance. Recently, a new trend in ELM emerges to combine it with ridge regression, which has been shown improved stability and generalization performance. However, this ridge parameter is determined through a trial-and-error manner, an unsatisfactory approach for automatic learning applications. In this paper, the differences between ridge ELM and ordinary Neural Networks are discussed as well as special properties of ridge ELM and various approaches to derive the ridge parameter. Furthermore, a semi-cross-validation ridge parameter selection procedure based on the special properties of ridge ELM is proposed. This approach, termed as Semi-Cross-validation Ridge ELM (SC-R-ELM), is also demonstrated to achieve robust and reliable results in 11 regression data sets.