Concentration of Measure and Global Optimization of Bayesian Multilayer Perceptron. Part I

Abstract

We present the description of a quasi-spherical coordinate system that is introduced in a space of parameters of a multilayer perceptron with ReLU and Leaky ReLU activation functions. In this instance, a regression loss function that is given in these coordinates becomes the sum of functions that depend on a set of functions defined on a sphere and a quasi-radial coordinate. Conditions for a concentration of measure are satisfied for the functions on the sphere. As a number of parameters tends to infinity, these criteria cause the loss function to concentrate toward a quasi-radially symmetric function.