Tuning of learning rate and momentum on back-propagation

Abstract

It is known well that backpropagation is used in recognition and learning on neural networks. The backpropagation, modification of the weight is calculated by learning rate ( eta =0.2) and momentum ( alpha =0.9). The number of training cycles depends on eta and alpha , so that it is necessary to choose the most suitable values for eta and alpha . Then, changing eta and alpha , the authors tried to search for the most suitable values for the learning. As a result, the combination of eta and alpha given the minimum value of the number of training cycles behave under the constant rule. Thus eta =K(1- alpha ). Moreover, the constant K is decided by the ratio between the number of output units and hidden units or the initialized weight.<>