Preprocessing of training set for backpropagation algorithm: histogram equalization

Abstract

This paper introduces a data preprocessing algorithm that can improve the efficiency of the standard backpropagation (BP) algorithm. The basic approach is transforming input data to a range that associates high-slopes of sigmoid where relatively large modification of weights occurs. This helps escaping of early trapping from prematured saturation. However, a simple and uniform transformation to such desired range can lead to a slow learning if the data have a heavily skewed distribution. In order to improve the performance of BP algorithm on such distribution, the authors propose a modified histogram equalization technique which enhances the spacing between data points in the heavily concentrated regions of skewed distribution. The authors' simulation study shows that this modified histogram equalization can significantly speed up the BP training as well as improving the generalization capability of the trained network.<>