Hybrid algorithm for training feed-forward neural networks using PSO-information gain with back propagation algorithm

Abstract

This paper proposes a hybrid algorithm for training a feed-forward neural network by combining both Particle Swarm Optimization (PSO) and Information Gain with Backpropagation (BP) algorithm. A conventional neural network training algorithm, i.e. BP, has several drawbacks in its slow convergence and local optima. Although PSO can be applied to search for the near optimal set of weights in the neural network, it may still stuck in the local optima because its fitness function depends merely on the error of the network. By combining the information gain of attributes in the dataset with the fitness function of PSO to train weights in the neural network, we find out that the resulting network has a significant improvement on its recognition rate. The comparisons among other training algorithm on two real-world datasets are provided and discussed.