{"title":"Concentration of Measure and Global Optimization of Bayesian Multilayer Perceptron. Part I","authors":"B. K. Temyanov, R. R. Nigmatullin","doi":"10.1134/s1995080224600651","DOIUrl":null,"url":null,"abstract":"<h3 data-test=\"abstract-sub-heading\">Abstract</h3><p>We present the description of a quasi-spherical coordinate system that is introduced in a space of parameters of a multilayer perceptron with ReLU and Leaky ReLU activation functions. In this instance, a regression loss function that is given in these coordinates becomes the sum of functions that depend on a set of functions defined on a sphere and a quasi-radial coordinate. Conditions for a concentration of measure are satisfied for the functions on the sphere. As a number of parameters tends to infinity, these criteria cause the loss function to concentrate toward a quasi-radially symmetric function.</p>","PeriodicalId":46135,"journal":{"name":"Lobachevskii Journal of Mathematics","volume":"22 1","pages":""},"PeriodicalIF":0.8000,"publicationDate":"2024-07-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Lobachevskii Journal of Mathematics","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1134/s1995080224600651","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"MATHEMATICS","Score":null,"Total":0}
引用次数: 0
Abstract
We present the description of a quasi-spherical coordinate system that is introduced in a space of parameters of a multilayer perceptron with ReLU and Leaky ReLU activation functions. In this instance, a regression loss function that is given in these coordinates becomes the sum of functions that depend on a set of functions defined on a sphere and a quasi-radial coordinate. Conditions for a concentration of measure are satisfied for the functions on the sphere. As a number of parameters tends to infinity, these criteria cause the loss function to concentrate toward a quasi-radially symmetric function.
期刊介绍：
Lobachevskii Journal of Mathematics is an international peer reviewed journal published in collaboration with the Russian Academy of Sciences and Kazan Federal University. The journal covers mathematical topics associated with the name of famous Russian mathematician Nikolai Lobachevsky (Lobachevskii). The journal publishes research articles on geometry and topology, algebra, complex analysis, functional analysis, differential equations and mathematical physics, probability theory and stochastic processes, computational mathematics, mathematical modeling, numerical methods and program complexes, computer science, optimal control, and theory of algorithms as well as applied mathematics. The journal welcomes manuscripts from all countries in the English language.
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
