Abdoulaye Koroko, Ani Anciaux-Sedrakian, Ibtihel Ben Gharbia, Valérie Garès, Mounir Haddou, Quang Huy Tran
{"title":"Analysis and comparison of two-level KFAC methods for training deep neural networks","authors":"Abdoulaye Koroko, Ani Anciaux-Sedrakian, Ibtihel Ben Gharbia, Valérie Garès, Mounir Haddou, Quang Huy Tran","doi":"10.1080/10556788.2024.2380684","DOIUrl":null,"url":null,"abstract":"As a second-order method, the Natural Gradient Descent (NGD) has the ability to accelerate training of neural networks. However, due to the prohibitive computational and memory costs of computing a...","PeriodicalId":54673,"journal":{"name":"Optimization Methods & Software","volume":null,"pages":null},"PeriodicalIF":1.4000,"publicationDate":"2024-07-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Optimization Methods & Software","FirstCategoryId":"5","ListUrlMain":"https://doi.org/10.1080/10556788.2024.2380684","RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"COMPUTER SCIENCE, SOFTWARE ENGINEERING","Score":null,"Total":0}
引用次数: 0
Abstract
As a second-order method, the Natural Gradient Descent (NGD) has the ability to accelerate training of neural networks. However, due to the prohibitive computational and memory costs of computing a...
期刊介绍：
Optimization Methods and Software
publishes refereed papers on the latest developments in the theory and realization of optimization methods, with particular emphasis on the interface between software development and algorithm design.
Topics include:
Theory, implementation and performance evaluation of algorithms and computer codes for linear, nonlinear, discrete, stochastic optimization and optimal control. This includes in particular conic, semi-definite, mixed integer, network, non-smooth, multi-objective and global optimization by deterministic or nondeterministic algorithms.
Algorithms and software for complementarity, variational inequalities and equilibrium problems, and also for solving inverse problems, systems of nonlinear equations and the numerical study of parameter dependent operators.
Various aspects of efficient and user-friendly implementations: e.g. automatic differentiation, massively parallel optimization, distributed computing, on-line algorithms, error sensitivity and validity analysis, problem scaling, stopping criteria and symbolic numeric interfaces.
Theoretical studies with clear potential for applications and successful applications of specially adapted optimization methods and software to fields like engineering, machine learning, data mining, economics, finance, biology, or medicine. These submissions should not consist solely of the straightforward use of standard optimization techniques.
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
