Investigation of the Transition Mechanism to Chaos in Multilayer Neural Networks

Abstract

Multilayer neural networks were considered. The program in the Python software environment was developed. The transition of the network to an unpredictable (chaotic) mode was described by a cascade of bifurcations that follow one another. The chaos was considered as consequence of the increase in the number of local minima in the area of the global minimum. Increasing of the number of layers of the neural network as well as the number of neurons in the hidden layer leads to an increase in the number of local minima when approaching the global minimum, and thus narrows the range of optimal solutions. Fractal structures on bifurcation diagram were observed. Therefore, the solution to the problem of automatic selection of the learning rate is related to the number of existing local minima.