SIAM Journal on Optimization, Volume 34, Issue 3, Page 3163-3166, September 2024. Abstract. As it is formulated, Proposition 3.12 of [M. Brokate and M. Ulbrich, SIAM J. Optim., 32 (2022), pp. 1265–1287] contains an error. But this can be corrected in the way described below. The results of [M. Brokate and M. Ulbrich, SIAM J. Optim., 32 (2022), pp. 1265–1287] based on Proposition 3.12 are not affected. We also use the opportunity to add a further illustrating example and to rectify some inaccuracies which may be confusing.
SIAM 优化期刊》，第 34 卷第 3 期，第 3163-3166 页，2024 年 9 月。 摘要M. Brokate and M. Ulbrich, SIAM J. Optim., 32 (2022), pp.但这可以通过下文所述的方法加以纠正。基于命题 3.12 的 [M. Brokate and M. Ulbrich, SIAM J. Optim.我们还借此机会增加了一个示例，并纠正了一些可能引起混淆的不准确之处。
{"title":"Corrigendum and Addendum: Newton Differentiability of Convex Functions in Normed Spaces and of a Class of Operators","authors":"Martin Brokate, Michael Ulbrich","doi":"10.1137/24m1669542","DOIUrl":"https://doi.org/10.1137/24m1669542","url":null,"abstract":"SIAM Journal on Optimization, Volume 34, Issue 3, Page 3163-3166, September 2024. <br/> Abstract. As it is formulated, Proposition 3.12 of [M. Brokate and M. Ulbrich, SIAM J. Optim., 32 (2022), pp. 1265–1287] contains an error. But this can be corrected in the way described below. The results of [M. Brokate and M. Ulbrich, SIAM J. Optim., 32 (2022), pp. 1265–1287] based on Proposition 3.12 are not affected. We also use the opportunity to add a further illustrating example and to rectify some inaccuracies which may be confusing.","PeriodicalId":49529,"journal":{"name":"SIAM Journal on Optimization","volume":null,"pages":null},"PeriodicalIF":3.1,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142251646","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Optimization, Volume 34, Issue 3, Page 3136-3162, September 2024. Abstract. The computation of the ground state of special multicomponent Bose–Einstein condensates (BECs) can be formulated as an energy functional minimization problem with spherical constraints. It leads to a nonconvex quartic-quadratic optimization problem after suitable discretizations. First, we generalize the Newton-based methods for single-component BECs to the alternating minimization scheme for multicomponent BECs. Second, the global convergent alternating Newton-Noda iteration (ANNI) is proposed. In particular, we prove the positivity preserving property of ANNI under mild conditions. Finally, our analysis is applied to a class of more general “multiblock” optimization problems with spherical constraints. Numerical experiments are performed to evaluate the performance of proposed methods for different multicomponent BECs, including pseudo spin-1/2, antiferromagnetic spin-1 and spin-2 BECs. These results support our theory and demonstrate the efficiency of our algorithms.
{"title":"Newton-Based Alternating Methods for the Ground State of a Class of Multicomponent Bose–Einstein Condensates","authors":"Pengfei Huang, Qingzhi Yang","doi":"10.1137/23m1580346","DOIUrl":"https://doi.org/10.1137/23m1580346","url":null,"abstract":"SIAM Journal on Optimization, Volume 34, Issue 3, Page 3136-3162, September 2024. <br/> Abstract. The computation of the ground state of special multicomponent Bose–Einstein condensates (BECs) can be formulated as an energy functional minimization problem with spherical constraints. It leads to a nonconvex quartic-quadratic optimization problem after suitable discretizations. First, we generalize the Newton-based methods for single-component BECs to the alternating minimization scheme for multicomponent BECs. Second, the global convergent alternating Newton-Noda iteration (ANNI) is proposed. In particular, we prove the positivity preserving property of ANNI under mild conditions. Finally, our analysis is applied to a class of more general “multiblock” optimization problems with spherical constraints. Numerical experiments are performed to evaluate the performance of proposed methods for different multicomponent BECs, including pseudo spin-1/2, antiferromagnetic spin-1 and spin-2 BECs. These results support our theory and demonstrate the efficiency of our algorithms.","PeriodicalId":49529,"journal":{"name":"SIAM Journal on Optimization","volume":null,"pages":null},"PeriodicalIF":3.1,"publicationDate":"2024-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142212452","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Christopher T. Ryan, Robert L. Smith, Marina A. Epelman
SIAM Journal on Optimization, Volume 34, Issue 3, Page 3112-3135, September 2024. Abstract. We discuss finding minimum spanning trees (MSTs) on connected graphs with countably many nodes of finite degree. When edge costs are summable and an MST exists (which is not guaranteed in general), we show that an algorithm that finds MSTs on finite subgraphs (called layers) converges in objective value to the cost of an MST of the whole graph as the sizes of the layers grow to infinity. We call this the layered greedy algorithm since a greedy algorithm is used to find MSTs on each finite layer. We stress that the overall algorithm is not greedy since edges can enter and leave iterate spanning trees as larger layers are considered. However, in the setting where the underlying graph has the finite cycle (FC) property (meaning that every edge is contained in at most finitely many cycles) and distinct edge costs, we show that a unique MST [math] exists and the layered greedy algorithm produces iterates that converge to [math] by eventually “locking in" edges after finitely many iterations. Applications to network deployment are discussed.
{"title":"Minimum Spanning Trees in Infinite Graphs: Theory and Algorithms","authors":"Christopher T. Ryan, Robert L. Smith, Marina A. Epelman","doi":"10.1137/23m157627x","DOIUrl":"https://doi.org/10.1137/23m157627x","url":null,"abstract":"SIAM Journal on Optimization, Volume 34, Issue 3, Page 3112-3135, September 2024. <br/> Abstract. We discuss finding minimum spanning trees (MSTs) on connected graphs with countably many nodes of finite degree. When edge costs are summable and an MST exists (which is not guaranteed in general), we show that an algorithm that finds MSTs on finite subgraphs (called layers) converges in objective value to the cost of an MST of the whole graph as the sizes of the layers grow to infinity. We call this the layered greedy algorithm since a greedy algorithm is used to find MSTs on each finite layer. We stress that the overall algorithm is not greedy since edges can enter and leave iterate spanning trees as larger layers are considered. However, in the setting where the underlying graph has the finite cycle (FC) property (meaning that every edge is contained in at most finitely many cycles) and distinct edge costs, we show that a unique MST [math] exists and the layered greedy algorithm produces iterates that converge to [math] by eventually “locking in\" edges after finitely many iterations. Applications to network deployment are discussed.","PeriodicalId":49529,"journal":{"name":"SIAM Journal on Optimization","volume":null,"pages":null},"PeriodicalIF":3.1,"publicationDate":"2024-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142212453","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Optimization, Volume 34, Issue 3, Page 3064-3087, September 2024. Abstract. We consider stochastic optimization problems involving an expected value of a nonlinear function of a base random vector and a conditional expectation of another function depending on the base random vector, a dependent random vector, and the decision variables. We call such problems conditional stochastic optimization problems. They arise in many applications, such as uplift modeling, reinforcement learning, and contextual optimization. We propose a specialized single time-scale stochastic method for nonconvex constrained conditional stochastic optimization problems with a Lipschitz smooth outer function and a generalized differentiable inner function. In the method, we approximate the inner conditional expectation with a rich parametric model whose mean squared error satisfies a stochastic version of a Łojasiewicz condition. The model is used by an inner learning algorithm. The main feature of our approach is that unbiased stochastic estimates of the directions used by the method can be generated with one observation from the joint distribution per iteration, which makes it applicable to real-time learning. The directions, however, are not gradients or subgradients of any overall objective function. We prove the convergence of the method with probability one, using the method of differential inclusions and a specially designed Lyapunov function, involving a stochastic generalization of the Bregman distance. Finally, a numerical illustration demonstrates the viability of our approach.
{"title":"A Functional Model Method for Nonconvex Nonsmooth Conditional Stochastic Optimization","authors":"Andrzej Ruszczyński, Shangzhe Yang","doi":"10.1137/23m1617965","DOIUrl":"https://doi.org/10.1137/23m1617965","url":null,"abstract":"SIAM Journal on Optimization, Volume 34, Issue 3, Page 3064-3087, September 2024. <br/> Abstract. We consider stochastic optimization problems involving an expected value of a nonlinear function of a base random vector and a conditional expectation of another function depending on the base random vector, a dependent random vector, and the decision variables. We call such problems conditional stochastic optimization problems. They arise in many applications, such as uplift modeling, reinforcement learning, and contextual optimization. We propose a specialized single time-scale stochastic method for nonconvex constrained conditional stochastic optimization problems with a Lipschitz smooth outer function and a generalized differentiable inner function. In the method, we approximate the inner conditional expectation with a rich parametric model whose mean squared error satisfies a stochastic version of a Łojasiewicz condition. The model is used by an inner learning algorithm. The main feature of our approach is that unbiased stochastic estimates of the directions used by the method can be generated with one observation from the joint distribution per iteration, which makes it applicable to real-time learning. The directions, however, are not gradients or subgradients of any overall objective function. We prove the convergence of the method with probability one, using the method of differential inclusions and a specially designed Lyapunov function, involving a stochastic generalization of the Bregman distance. Finally, a numerical illustration demonstrates the viability of our approach.","PeriodicalId":49529,"journal":{"name":"SIAM Journal on Optimization","volume":null,"pages":null},"PeriodicalIF":3.1,"publicationDate":"2024-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142212455","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Optimization, Volume 34, Issue 3, Page 3088-3111, September 2024. Abstract. In this paper, we analyze minimal representations of [math]-power cones as simpler cones. We derive some new results on the complexity of the representations, and we provide a procedure to construct a minimal representation by means of second order cones in case [math] and [math] are rational. The construction is based on the identification of the cones with a graph, the mediated graph. Then, we develop a mixed integer linear optimization formulation to obtain the optimal mediated graph, and then the minimal representation. We present the results of a series of computational experiments in order to analyze the computational performance of the approach, both to obtain the representation and its incorporation into a practical conic optimization model that arises in facility location.
{"title":"On Minimal Extended Representations of Generalized Power Cones","authors":"Víctor Blanco, Miguel Martínez-Antón","doi":"10.1137/23m1617205","DOIUrl":"https://doi.org/10.1137/23m1617205","url":null,"abstract":"SIAM Journal on Optimization, Volume 34, Issue 3, Page 3088-3111, September 2024. <br/> Abstract. In this paper, we analyze minimal representations of [math]-power cones as simpler cones. We derive some new results on the complexity of the representations, and we provide a procedure to construct a minimal representation by means of second order cones in case [math] and [math] are rational. The construction is based on the identification of the cones with a graph, the mediated graph. Then, we develop a mixed integer linear optimization formulation to obtain the optimal mediated graph, and then the minimal representation. We present the results of a series of computational experiments in order to analyze the computational performance of the approach, both to obtain the representation and its incorporation into a practical conic optimization model that arises in facility location.","PeriodicalId":49529,"journal":{"name":"SIAM Journal on Optimization","volume":null,"pages":null},"PeriodicalIF":3.1,"publicationDate":"2024-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142212454","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Nizar Bousselmi, Julien M. Hendrickx, François Glineur
SIAM Journal on Optimization, Volume 34, Issue 3, Page 3033-3063, September 2024. Abstract. The performance estimation problem methodology makes it possible to determine the exact worst-case performance of an optimization method. In this work, we generalize this framework to first-order methods involving linear operators. This extension requires an explicit formulation of interpolation conditions for those linear operators. We consider the class of linear operators [math], where matrix [math] has bounded singular values, and the class of linear operators, where [math] is symmetric and has bounded eigenvalues. We describe interpolation conditions for these classes, i.e., necessary and sufficient conditions that, given a list of pairs [math], characterize the existence of a linear operator mapping [math] to [math] for all [math]. Using these conditions, we first identify the exact worst-case behavior of the gradient method applied to the composed objective [math], and observe that it always corresponds to [math] being a scaling operator. We then investigate the Chambolle–Pock method applied to [math], and improve the existing analysis to obtain a proof of the exact convergence rate of the primal-dual gap. In addition, we study how this method behaves on Lipschitz convex functions, and obtain a numerical convergence rate for the primal accuracy of the last iterate. We also show numerically that averaging iterates is beneficial in this setting.
{"title":"Interpolation Conditions for Linear Operators and Applications to Performance Estimation Problems","authors":"Nizar Bousselmi, Julien M. Hendrickx, François Glineur","doi":"10.1137/23m1575391","DOIUrl":"https://doi.org/10.1137/23m1575391","url":null,"abstract":"SIAM Journal on Optimization, Volume 34, Issue 3, Page 3033-3063, September 2024. <br/> Abstract. The performance estimation problem methodology makes it possible to determine the exact worst-case performance of an optimization method. In this work, we generalize this framework to first-order methods involving linear operators. This extension requires an explicit formulation of interpolation conditions for those linear operators. We consider the class of linear operators [math], where matrix [math] has bounded singular values, and the class of linear operators, where [math] is symmetric and has bounded eigenvalues. We describe interpolation conditions for these classes, i.e., necessary and sufficient conditions that, given a list of pairs [math], characterize the existence of a linear operator mapping [math] to [math] for all [math]. Using these conditions, we first identify the exact worst-case behavior of the gradient method applied to the composed objective [math], and observe that it always corresponds to [math] being a scaling operator. We then investigate the Chambolle–Pock method applied to [math], and improve the existing analysis to obtain a proof of the exact convergence rate of the primal-dual gap. In addition, we study how this method behaves on Lipschitz convex functions, and obtain a numerical convergence rate for the primal accuracy of the last iterate. We also show numerically that averaging iterates is beneficial in this setting.","PeriodicalId":49529,"journal":{"name":"SIAM Journal on Optimization","volume":null,"pages":null},"PeriodicalIF":3.1,"publicationDate":"2024-09-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142212456","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Optimization, Volume 34, Issue 3, Page 3005-3032, September 2024. Abstract. This paper develops and analyzes an accelerated proximal descent method for finding stationary points of nonconvex composite optimization problems. The objective function is of the form [math], where [math] is a proper closed convex function, [math] is a differentiable function on the domain of [math], and [math] is Lipschitz continuous on the domain of [math]. The main advantage of this method is that it is “parameter-free” in the sense that it does not require knowledge of the Lipschitz constant of [math] or of any global topological properties of [math]. It is shown that the proposed method can obtain an [math]-approximate stationary point with iteration complexity bounds that are optimal, up to logarithmic terms over [math], in both the convex and nonconvex settings. Some discussion is also given about how the proposed method can be leveraged in other existing optimization frameworks, such as min-max smoothing and penalty frameworks for constrained programming, to create more specialized parameter-free methods. Finally, numerical experiments are presented to support the practical viability of the method.
{"title":"Complexity-Optimal and Parameter-Free First-Order Methods for Finding Stationary Points of Composite Optimization Problems","authors":"Weiwei Kong","doi":"10.1137/22m1498826","DOIUrl":"https://doi.org/10.1137/22m1498826","url":null,"abstract":"SIAM Journal on Optimization, Volume 34, Issue 3, Page 3005-3032, September 2024. <br/> Abstract. This paper develops and analyzes an accelerated proximal descent method for finding stationary points of nonconvex composite optimization problems. The objective function is of the form [math], where [math] is a proper closed convex function, [math] is a differentiable function on the domain of [math], and [math] is Lipschitz continuous on the domain of [math]. The main advantage of this method is that it is “parameter-free” in the sense that it does not require knowledge of the Lipschitz constant of [math] or of any global topological properties of [math]. It is shown that the proposed method can obtain an [math]-approximate stationary point with iteration complexity bounds that are optimal, up to logarithmic terms over [math], in both the convex and nonconvex settings. Some discussion is also given about how the proposed method can be leveraged in other existing optimization frameworks, such as min-max smoothing and penalty frameworks for constrained programming, to create more specialized parameter-free methods. Finally, numerical experiments are presented to support the practical viability of the method.","PeriodicalId":49529,"journal":{"name":"SIAM Journal on Optimization","volume":null,"pages":null},"PeriodicalIF":3.1,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142212457","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Optimization, Volume 34, Issue 3, Page 2943-2972, September 2024. Abstract. We propose a new method for linear second-order cone programs. It is based on the sequential quadratic programming framework for nonlinear programming. In contrast to interior point methods, it can capitalize on the warm-start capabilities of active-set quadratic programming subproblem solvers and achieve a local quadratic rate of convergence. In order to overcome the nondifferentiability or singularity observed in nonlinear formulations of the conic constraints, the subproblems approximate the cones with polyhedral outer approximations that are refined throughout the iterations. For nondegenerate instances, the algorithm implicitly identifies the set of cones for which the optimal solution lies at the extreme points. As a consequence, the final steps are identical to regular sequential quadratic programming steps for a differentiable nonlinear optimization problem, yielding local quadratic convergence. We prove the global and local convergence guarantees of the method and present numerical experiments that confirm that the method can take advantage of good starting points and can achieve higher accuracy compared to a state-of-the-art interior point solver.
{"title":"A Quadratically Convergent Sequential Programming Method for Second-Order Cone Programs Capable of Warm Starts","authors":"Xinyi Luo, Andreas Wächter","doi":"10.1137/22m1507681","DOIUrl":"https://doi.org/10.1137/22m1507681","url":null,"abstract":"SIAM Journal on Optimization, Volume 34, Issue 3, Page 2943-2972, September 2024. <br/> Abstract. We propose a new method for linear second-order cone programs. It is based on the sequential quadratic programming framework for nonlinear programming. In contrast to interior point methods, it can capitalize on the warm-start capabilities of active-set quadratic programming subproblem solvers and achieve a local quadratic rate of convergence. In order to overcome the nondifferentiability or singularity observed in nonlinear formulations of the conic constraints, the subproblems approximate the cones with polyhedral outer approximations that are refined throughout the iterations. For nondegenerate instances, the algorithm implicitly identifies the set of cones for which the optimal solution lies at the extreme points. As a consequence, the final steps are identical to regular sequential quadratic programming steps for a differentiable nonlinear optimization problem, yielding local quadratic convergence. We prove the global and local convergence guarantees of the method and present numerical experiments that confirm that the method can take advantage of good starting points and can achieve higher accuracy compared to a state-of-the-art interior point solver.","PeriodicalId":49529,"journal":{"name":"SIAM Journal on Optimization","volume":null,"pages":null},"PeriodicalIF":3.1,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142212458","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Optimization, Volume 34, Issue 3, Page 2883-2916, September 2024. Abstract. It is well known that there have been many numerical algorithms for solving nonsmooth minimax problems; however, numerical algorithms for nonsmooth minimax problems with joint linear constraints are very rare. This paper aims to discuss optimality conditions and develop practical numerical algorithms for minimax problems with joint linear constraints. First, we use the properties of proximal mapping and the KKT system to establish optimality conditions. Second, we propose a framework of an alternating coordinate algorithm for the minimax problem and analyze its convergence properties. Third, we develop a proximal gradient multistep ascent descent method (PGmsAD) as a numerical algorithm and demonstrate that the method can find an [math]-stationary point for this kind of nonsmooth problem in [math] iterations. Finally, we apply PGmsAD to generalized absolute value equations, generalized linear projection equations, and linear regression problems, and we report the efficiency of PGmsAD on large-scale optimization.
{"title":"Optimality Conditions and Numerical Algorithms for a Class of Linearly Constrained Minimax Optimization Problems","authors":"Yu-Hong Dai, Jiani Wang, Liwei Zhang","doi":"10.1137/22m1535243","DOIUrl":"https://doi.org/10.1137/22m1535243","url":null,"abstract":"SIAM Journal on Optimization, Volume 34, Issue 3, Page 2883-2916, September 2024. <br/> Abstract. It is well known that there have been many numerical algorithms for solving nonsmooth minimax problems; however, numerical algorithms for nonsmooth minimax problems with joint linear constraints are very rare. This paper aims to discuss optimality conditions and develop practical numerical algorithms for minimax problems with joint linear constraints. First, we use the properties of proximal mapping and the KKT system to establish optimality conditions. Second, we propose a framework of an alternating coordinate algorithm for the minimax problem and analyze its convergence properties. Third, we develop a proximal gradient multistep ascent descent method (PGmsAD) as a numerical algorithm and demonstrate that the method can find an [math]-stationary point for this kind of nonsmooth problem in [math] iterations. Finally, we apply PGmsAD to generalized absolute value equations, generalized linear projection equations, and linear regression problems, and we report the efficiency of PGmsAD on large-scale optimization.","PeriodicalId":49529,"journal":{"name":"SIAM Journal on Optimization","volume":null,"pages":null},"PeriodicalIF":3.1,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142212460","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Optimization, Volume 34, Issue 3, Page 2973-3004, September 2024. Abstract. In this paper we study consensus-based optimization (CBO), which is a multiagent metaheuristic derivative-free optimization method that can globally minimize nonconvex nonsmooth functions and is amenable to theoretical analysis. Based on an experimentally supported intuition that, on average, CBO performs a gradient descent of the squared Euclidean distance to the global minimizer, we devise a novel technique for proving the convergence to the global minimizer in mean-field law for a rich class of objective functions. The result unveils internal mechanisms of CBO that are responsible for the success of the method. In particular, we prove that CBO performs a convexification of a large class of optimization problems as the number of optimizing agents goes to infinity. Furthermore, we improve prior analyses by requiring mild assumptions about the initialization of the method and by covering objectives that are merely locally Lipschitz continuous. As a core component of this analysis, we establish a quantitative nonasymptotic Laplace principle, which may be of independent interest. From the result of CBO convergence in mean-field law, it becomes apparent that the hardness of any global optimization problem is necessarily encoded in the rate of the mean-field approximation, for which we provide a novel probabilistic quantitative estimate. The combination of these results allows us to obtain probabilistic global convergence guarantees of the numerical CBO method.
{"title":"Consensus-Based Optimization Methods Converge Globally","authors":"Massimo Fornasier, Timo Klock, Konstantin Riedl","doi":"10.1137/22m1527805","DOIUrl":"https://doi.org/10.1137/22m1527805","url":null,"abstract":"SIAM Journal on Optimization, Volume 34, Issue 3, Page 2973-3004, September 2024. <br/> Abstract. In this paper we study consensus-based optimization (CBO), which is a multiagent metaheuristic derivative-free optimization method that can globally minimize nonconvex nonsmooth functions and is amenable to theoretical analysis. Based on an experimentally supported intuition that, on average, CBO performs a gradient descent of the squared Euclidean distance to the global minimizer, we devise a novel technique for proving the convergence to the global minimizer in mean-field law for a rich class of objective functions. The result unveils internal mechanisms of CBO that are responsible for the success of the method. In particular, we prove that CBO performs a convexification of a large class of optimization problems as the number of optimizing agents goes to infinity. Furthermore, we improve prior analyses by requiring mild assumptions about the initialization of the method and by covering objectives that are merely locally Lipschitz continuous. As a core component of this analysis, we establish a quantitative nonasymptotic Laplace principle, which may be of independent interest. From the result of CBO convergence in mean-field law, it becomes apparent that the hardness of any global optimization problem is necessarily encoded in the rate of the mean-field approximation, for which we provide a novel probabilistic quantitative estimate. The combination of these results allows us to obtain probabilistic global convergence guarantees of the numerical CBO method.","PeriodicalId":49529,"journal":{"name":"SIAM Journal on Optimization","volume":null,"pages":null},"PeriodicalIF":3.1,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142226831","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号