On the convergence of a block-gradient algorithm for back-propagation learning

Abstract

A block-gradient algorithm is defined, where the weight matrix is updated after every presentation of a block of b examples each. Total and stochastic gradients are included in the block-gradient algorithm, for particular values of b. Experimental laws are stated on the speed of convergence, according to the block size. The first law indicates that an adaptive learning rate has to respect an exponential decreasing function of the number of examples presented between two successive weight updates. The second law states that, with an adaptive learning rate value, the number of epochs grows linearly with the size of the exemplar blocks. The last one shows how the number of epochs for reaching a given level of performance depends on the learning rate.<>