An Efficient Neural Network Training Algorithm with Maximized Gradient Function and Modulated Chaos

Abstract

Biological brain involves chaos and the structure of artificial neural networks (ANNs) is similar to human brain. In order to imitate the structure and the function of human brain better, it is more logical to combine chaos with neural networks. In this paper we proposed a chaotic learning algorithm called Maximized Gradient function and Modulated Chaos (MGMC). MGMC maximizes the gradient function and also added a modulated version of chaos in learning rate (LR) as well as in activation function. Activation function made adaptive by using chaos as gain factor. MGMC generates a chaotic time series as modulated form of Mackey Glass, Logistic Map and Lorenz Attractor. A rescaled version of this series is used as learning rate (LR) called Modulated Learning Rate (MLR) during NN training. As a result neural network becomes biologically plausible and may get escaped from local minima zone and faster convergence rate is obtained as maximizing the derivative of activation function together with minimizing the error function. MGMC is extensively tested on three real world benchmark classification problems such as australian credit card, wine and soybean identification. The proposed MGMC outperforms the existing BP and BPfast in terms of generalization ability and also convergence rate.