A hybrid learning algorithm for multilayer perceptrons to improve generalization under sparse training data conditions

Abstract

The backpropagation algorithm is mainly used for multilayer perceptrons. This algorithm is, however, difficult to achieve high generalization when the number of training data is limited, i.e. sparse training data. In this paper, a new learning algorithm is proposed. It combines the BP algorithm and modifies hyperplanes taking internal information into account. In other words, the hyperplanes are controlled by the distance between the hyperplanes and the critical training data, which locate close to the boundary. This algorithm works well for the sparse training data to achieve high generalization. In order to evaluate generalization, it is assumed that all data are normally distributed around the training data. Several simulations of pattern classification demonstrate the efficiency of the proposed algorithm.