Analysis of Weight Initialization Routines for Scaled Conjugate Gradient Training Algorithm

The choice of weight initialization routines is one of the important choices to be made for improving the training efficiency of an artificial neural network. In this paper, we analyze the affect of many known weight initialization routines, on training of an artificial neural network, when it was trained with a second order scaled conjugate gradient training algorithm. A number of experiments were conducted to perform this analysis over eight selected function approximation problems. The results suggest that the partially deterministic weight initialization method and the Nguyen-Widrow initialization technique performed equally well and helped the network train and generalize better by achieving better training and simulation error values.