Extreme learning machine with multiple kernels

Abstract

Recently a novel learning algorithm called extreme learning machine (ELM) was proposed for efficiently training single-hidden layer feedforward neural networks (SLFNs). Compared with other traditional gradient-descent-based learning algorithms, ELM has shown promising results because it chooses weights and biases of hidden nodes randomly and obtains the output weights and biases analytically. In most cases, ELM is fast and presents good generalization, but we find that the stability and generalization performance still can be improved. In this paper, we propose a hybrid model which combines the advantage of ELM and the advantage of Bayesian “sum of kernels” model, named Extreme Learning Machine with Multiple Kernels (MK-ELM). This method optimizes the kernel function using a weighted sum of kernel functions by a prior knowledge. Experimental results show that this approach is able to make neural networks more robust and generates better generalization performance for both regression and classification applications.