Yuchen Fang, Sen Na, Michael W. Mahoney, Mladen Kolar
SIAM Journal on Optimization, Volume 34, Issue 2, Page 2007-2037, June 2024. Abstract. We propose a trust-region stochastic sequential quadratic programming algorithm (TR-StoSQP) to solve nonlinear optimization problems with stochastic objectives and deterministic equality constraints. We consider a fully stochastic setting, where at each step a single sample is generated to estimate the objective gradient. The algorithm adaptively selects the trust-region radius and, compared to the existing line-search StoSQP schemes, allows us to utilize indefinite Hessian matrices (i.e., Hessians without modification) in SQP subproblems. As a trust-region method for constrained optimization, our algorithm must address an infeasibility issue—the linearized equality constraints and trust-region constraints may lead to infeasible SQP subproblems. In this regard, we propose an adaptive relaxation technique to compute the trial step, consisting of a normal step and a tangential step. To control the lengths of these two steps while ensuring a scale-invariant property, we adaptively decompose the trust-region radius into two segments, based on the proportions of the rescaled feasibility and optimality residuals to the rescaled full KKT residual. The normal step has a closed form, while the tangential step is obtained by solving a trust-region subproblem, to which a solution ensuring the Cauchy reduction is sufficient for our study. We establish a global almost sure convergence guarantee for TR-StoSQP and illustrate its empirical performance on both a subset of problems in the CUTEst test set and constrained logistic regression problems using data from the LIBSVM collection.
{"title":"Fully Stochastic Trust-Region Sequential Quadratic Programming for Equality-Constrained Optimization Problems","authors":"Yuchen Fang, Sen Na, Michael W. Mahoney, Mladen Kolar","doi":"10.1137/22m1537862","DOIUrl":"https://doi.org/10.1137/22m1537862","url":null,"abstract":"SIAM Journal on Optimization, Volume 34, Issue 2, Page 2007-2037, June 2024. <br/> Abstract. We propose a trust-region stochastic sequential quadratic programming algorithm (TR-StoSQP) to solve nonlinear optimization problems with stochastic objectives and deterministic equality constraints. We consider a fully stochastic setting, where at each step a single sample is generated to estimate the objective gradient. The algorithm adaptively selects the trust-region radius and, compared to the existing line-search StoSQP schemes, allows us to utilize indefinite Hessian matrices (i.e., Hessians without modification) in SQP subproblems. As a trust-region method for constrained optimization, our algorithm must address an infeasibility issue—the linearized equality constraints and trust-region constraints may lead to infeasible SQP subproblems. In this regard, we propose an adaptive relaxation technique to compute the trial step, consisting of a normal step and a tangential step. To control the lengths of these two steps while ensuring a scale-invariant property, we adaptively decompose the trust-region radius into two segments, based on the proportions of the rescaled feasibility and optimality residuals to the rescaled full KKT residual. The normal step has a closed form, while the tangential step is obtained by solving a trust-region subproblem, to which a solution ensuring the Cauchy reduction is sufficient for our study. We establish a global almost sure convergence guarantee for TR-StoSQP and illustrate its empirical performance on both a subset of problems in the CUTEst test set and constrained logistic regression problems using data from the LIBSVM collection.","PeriodicalId":49529,"journal":{"name":"SIAM Journal on Optimization","volume":null,"pages":null},"PeriodicalIF":3.1,"publicationDate":"2024-06-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141500630","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Optimization, Volume 34, Issue 2, Page 1970-2006, June 2024. Abstract. A nonnegativity certificate (NNC) is a way to write a polynomial so that its nonnegativity on a semialgebraic set becomes evident. Positivstellensätze (Psätze) guarantee the existence of NNCs. Both NNCs and Psätze underlie powerful algorithmic techniques for optimization. This paper proposes a universal approach to derive new Psätze for general semialgebraic sets from ones developed for simpler sets, such as a box, a simplex, or the nonnegative orthant. We provide several results illustrating the approach. First, by considering Handelman’s Positivstellensatz (Psatz) over a box, we construct non-SOS Schmüdgen-type Psätze over any compact semialgebraic set, that is, a family of Psätze that follow the structure of the fundamental Schmüdgen’s Psatz but where instead of SOS polynomials, any class of polynomials containing the nonnegative constants can be used, such as SONC, DSOS/SDSOS, hyperbolic, or sums of AM/GM polynomials. Second, by considering the simplex as the simple set, we derive a sparse Psatz over general compact sets which does not rely on any structural assumptions of the set. Finally, by considering Pólya’s Psatz over the nonnegative orthant, we derive a new non-SOS Psatz over unbounded sets which satisfy some generic conditions. All these results contribute to the literature regarding the use of non-SOS polynomials and sparse NNCs to derive Psätze over compact and unbounded sets. Throughout the article, we illustrate our results with relevant examples and numerical experiments.
SIAM 优化期刊》，第 34 卷第 2 期，第 1970-2006 页，2024 年 6 月。 摘要。非负性证明（NNC）是多项式的一种写法，它使多项式在半代数集合上的非负性变得明显。Positivstellensätze (Psätze) 保证 NNCs 的存在。NNCs和Psätze都是强大的优化算法技术的基础。本文提出了一种通用方法，即从针对较简单集合（如盒形集合、单纯形集合或非负正交集合）开发的新Psätze推导出针对一般半代数集合的新Psätze。我们提供了几个结果来说明这种方法。首先，通过考虑方格上的汉德尔曼正定定理（Psatz），我们构建了任何紧凑半代数集合上的非 SOS 施密特型定理，即遵循基本施密特定理结构的定理族，但其中可以使用任何包含非负常数的多项式类代替 SOS 多项式，如 SONC、DSOS/SDSOS、双曲线或 AM/GM 多项式之和。其次，通过将单纯形视为单纯集，我们推导出了一般单纯集上的稀疏 Psatz，它不依赖于单纯集的任何结构假设。最后，通过考虑非负正交上的 Pólya Psatz，我们推导出了无界集合上的一种新的非 SOS Psatz，它满足一些通用条件。所有这些结果都为有关使用非 SOS 多项式和稀疏 NNC 来推导紧凑集和无约束集上的 Psätze 的文献做出了贡献。在整篇文章中，我们用相关的例子和数值实验来说明我们的结果。
{"title":"Reducing Nonnegativity over General Semialgebraic Sets to Nonnegativity over Simple Sets","authors":"Olga Kuryatnikova, Juan C. Vera, Luis F. Zuluaga","doi":"10.1137/22m1501027","DOIUrl":"https://doi.org/10.1137/22m1501027","url":null,"abstract":"SIAM Journal on Optimization, Volume 34, Issue 2, Page 1970-2006, June 2024. <br/> Abstract. A nonnegativity certificate (NNC) is a way to write a polynomial so that its nonnegativity on a semialgebraic set becomes evident. Positivstellensätze (Psätze) guarantee the existence of NNCs. Both NNCs and Psätze underlie powerful algorithmic techniques for optimization. This paper proposes a universal approach to derive new Psätze for general semialgebraic sets from ones developed for simpler sets, such as a box, a simplex, or the nonnegative orthant. We provide several results illustrating the approach. First, by considering Handelman’s Positivstellensatz (Psatz) over a box, we construct non-SOS Schmüdgen-type Psätze over any compact semialgebraic set, that is, a family of Psätze that follow the structure of the fundamental Schmüdgen’s Psatz but where instead of SOS polynomials, any class of polynomials containing the nonnegative constants can be used, such as SONC, DSOS/SDSOS, hyperbolic, or sums of AM/GM polynomials. Second, by considering the simplex as the simple set, we derive a sparse Psatz over general compact sets which does not rely on any structural assumptions of the set. Finally, by considering Pólya’s Psatz over the nonnegative orthant, we derive a new non-SOS Psatz over unbounded sets which satisfy some generic conditions. All these results contribute to the literature regarding the use of non-SOS polynomials and sparse NNCs to derive Psätze over compact and unbounded sets. Throughout the article, we illustrate our results with relevant examples and numerical experiments.","PeriodicalId":49529,"journal":{"name":"SIAM Journal on Optimization","volume":null,"pages":null},"PeriodicalIF":3.1,"publicationDate":"2024-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141546441","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Optimization, Volume 34, Issue 2, Page 1937-1969, June 2024. Abstract. In this paper, we study a class of unconstrained and constrained bilevel optimization problems in which the lower level is a possibly nonsmooth convex optimization problem, while the upper level is a possibly nonconvex optimization problem. We introduce a notion of [math]-KKT solution for them and show that an [math]-KKT solution leads to an [math]- or [math]-hypergradient–based stationary point under suitable assumptions. We also propose first-order penalty methods for finding an [math]-KKT solution of them, whose subproblems turn out to be a structured minimax problem and can be suitably solved by a first-order method recently developed by the authors. Under suitable assumptions, an operation complexity of [math] and [math], measured by their fundamental operations, is established for the proposed penalty methods for finding an [math]-KKT solution of the unconstrained and constrained bilevel optimization problems, respectively. Preliminary numerical results are presented to illustrate the performance of our proposed methods. To the best of our knowledge, this paper is the first work to demonstrate that bilevel optimization can be approximately solved as minimax optimization, and moreover, it provides the first implementable method with complexity guarantees for such sophisticated bilevel optimization.
{"title":"First-Order Penalty Methods for Bilevel Optimization","authors":"Zhaosong Lu, Sanyou Mei","doi":"10.1137/23m1566753","DOIUrl":"https://doi.org/10.1137/23m1566753","url":null,"abstract":"SIAM Journal on Optimization, Volume 34, Issue 2, Page 1937-1969, June 2024. <br/> Abstract. In this paper, we study a class of unconstrained and constrained bilevel optimization problems in which the lower level is a possibly nonsmooth convex optimization problem, while the upper level is a possibly nonconvex optimization problem. We introduce a notion of [math]-KKT solution for them and show that an [math]-KKT solution leads to an [math]- or [math]-hypergradient–based stationary point under suitable assumptions. We also propose first-order penalty methods for finding an [math]-KKT solution of them, whose subproblems turn out to be a structured minimax problem and can be suitably solved by a first-order method recently developed by the authors. Under suitable assumptions, an operation complexity of [math] and [math], measured by their fundamental operations, is established for the proposed penalty methods for finding an [math]-KKT solution of the unconstrained and constrained bilevel optimization problems, respectively. Preliminary numerical results are presented to illustrate the performance of our proposed methods. To the best of our knowledge, this paper is the first work to demonstrate that bilevel optimization can be approximately solved as minimax optimization, and moreover, it provides the first implementable method with complexity guarantees for such sophisticated bilevel optimization.","PeriodicalId":49529,"journal":{"name":"SIAM Journal on Optimization","volume":null,"pages":null},"PeriodicalIF":3.1,"publicationDate":"2024-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141519784","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pierre Carpentier, Jean-Philippe Chancelier, Michel De Lara
SIAM Journal on Optimization, Volume 34, Issue 2, Page 1909-1936, June 2024. Abstract. We consider sequences—indexed by time (discrete stages)—of parametric families of multistage stochastic optimization problems; thus, at each time, the optimization problems in a family are parameterized by some quantities (initial states, constraint levels, and so on). In this framework, we introduce an adapted notion of parametric time-consistent optimal solutions: They are solutions that remain optimal after truncation of the past and that are optimal for any values of the parameters. We link this time consistency notion with the concept of state variable in Markov decision processes for a class of multistage stochastic optimization problems incorporating state constraints at the final time, formulated in expectation. For such problems, when the primitive noise random process is stagewise independent and takes a finite number of values, we show that time-consistent solutions can be obtained by considering a finite-dimensional state variable. We illustrate our results on a simple dam management problem.
{"title":"Time Consistency for Multistage Stochastic Optimization Problems under Constraints in Expectation","authors":"Pierre Carpentier, Jean-Philippe Chancelier, Michel De Lara","doi":"10.1137/22m151830x","DOIUrl":"https://doi.org/10.1137/22m151830x","url":null,"abstract":"SIAM Journal on Optimization, Volume 34, Issue 2, Page 1909-1936, June 2024. <br/> Abstract. We consider sequences—indexed by time (discrete stages)—of parametric families of multistage stochastic optimization problems; thus, at each time, the optimization problems in a family are parameterized by some quantities (initial states, constraint levels, and so on). In this framework, we introduce an adapted notion of parametric time-consistent optimal solutions: They are solutions that remain optimal after truncation of the past and that are optimal for any values of the parameters. We link this time consistency notion with the concept of state variable in Markov decision processes for a class of multistage stochastic optimization problems incorporating state constraints at the final time, formulated in expectation. For such problems, when the primitive noise random process is stagewise independent and takes a finite number of values, we show that time-consistent solutions can be obtained by considering a finite-dimensional state variable. We illustrate our results on a simple dam management problem.","PeriodicalId":49529,"journal":{"name":"SIAM Journal on Optimization","volume":null,"pages":null},"PeriodicalIF":3.1,"publicationDate":"2024-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141519785","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Optimization, Volume 34, Issue 2, Page 1879-1908, June 2024. Abstract. In this paper, we study zeroth-order algorithms for nonconvex-concave minimax problems, which have attracted much attention in machine learning, signal processing, and many other fields in recent years. We propose a zeroth-order alternating randomized gradient projection (ZO-AGP) algorithm for smooth nonconvex-concave minimax problems; its iteration complexity to obtain an [math]-stationary point is bounded by [math], and the number of function value estimates is bounded by [math] per iteration. Moreover, we propose a zeroth-order block alternating randomized proximal gradient algorithm (ZO-BAPG) for solving blockwise nonsmooth nonconvex-concave minimax optimization problems; its iteration complexity to obtain an [math]-stationary point is bounded by [math], and the number of function value estimates per iteration is bounded by [math]. To the best of our knowledge, this is the first time zeroth-order algorithms with iteration complexity guarantee are developed for solving both general smooth and blockwise nonsmooth nonconvex-concave minimax problems. Numerical results on the data poisoning attack problem and the distributed nonconvex sparse principal component analysis problem validate the efficiency of the proposed algorithms.
{"title":"Derivative-Free Alternating Projection Algorithms for General Nonconvex-Concave Minimax Problems","authors":"Zi Xu, Ziqi Wang, Jingjing Shen, Yuhong Dai","doi":"10.1137/23m1568168","DOIUrl":"https://doi.org/10.1137/23m1568168","url":null,"abstract":"SIAM Journal on Optimization, Volume 34, Issue 2, Page 1879-1908, June 2024. <br/> Abstract. In this paper, we study zeroth-order algorithms for nonconvex-concave minimax problems, which have attracted much attention in machine learning, signal processing, and many other fields in recent years. We propose a zeroth-order alternating randomized gradient projection (ZO-AGP) algorithm for smooth nonconvex-concave minimax problems; its iteration complexity to obtain an [math]-stationary point is bounded by [math], and the number of function value estimates is bounded by [math] per iteration. Moreover, we propose a zeroth-order block alternating randomized proximal gradient algorithm (ZO-BAPG) for solving blockwise nonsmooth nonconvex-concave minimax optimization problems; its iteration complexity to obtain an [math]-stationary point is bounded by [math], and the number of function value estimates per iteration is bounded by [math]. To the best of our knowledge, this is the first time zeroth-order algorithms with iteration complexity guarantee are developed for solving both general smooth and blockwise nonsmooth nonconvex-concave minimax problems. Numerical results on the data poisoning attack problem and the distributed nonconvex sparse principal component analysis problem validate the efficiency of the proposed algorithms.","PeriodicalId":49529,"journal":{"name":"SIAM Journal on Optimization","volume":null,"pages":null},"PeriodicalIF":3.1,"publicationDate":"2024-05-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141168332","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM Journal on Optimization, Volume 34, Issue 2, Page 1852-1878, June 2024. Abstract. In this article, we are interested in developing polynomial decomposition techniques based on sums-of-squares (SOS), namely the difference-of-sums-of-squares (D-SOS) and the difference-of-convex-sums-of-squares (DC-SOS). In particular, the DC-SOS decomposition is very useful for difference-of-convex (DC) programming formulation of polynomial optimization problems. First, we introduce the cone of convex-sums-of-squares (CSOS) polynomials and discuss its relationship to the sums-of-squares (SOS) polynomials, the non-negative polynomials, and the SOS-convex polynomials. Then we propose the set of D-SOS and DC-SOS polynomials and prove that any polynomial can be formulated as D-SOS and DC-SOS. The problem of finding D-SOS and DC-SOS decompositions can be formulated as a semi-definite program and solved for any desired precision in polynomial time using interior point methods. Some algebraic properties of CSOS, D-SOS, and DC-SOS are established. Second, we focus on establishing several practical algorithms for exact D-SOS and DC-SOS polynomial decompositions without solving any SDP. The numerical performance of the proposed D-SOS and DC-SOS decomposition algorithms and their parallel versions, tested on a dataset of 1750 randomly generated polynomials, is reported.
{"title":"On Difference-of-SOS and Difference-of-Convex-SOS Decompositions for Polynomials","authors":"Yi-Shuai Niu, Hoai An Le Thi, Dinh Tao Pham","doi":"10.1137/22m1495524","DOIUrl":"https://doi.org/10.1137/22m1495524","url":null,"abstract":"SIAM Journal on Optimization, Volume 34, Issue 2, Page 1852-1878, June 2024. <br/> Abstract. In this article, we are interested in developing polynomial decomposition techniques based on sums-of-squares (SOS), namely the difference-of-sums-of-squares (D-SOS) and the difference-of-convex-sums-of-squares (DC-SOS). In particular, the DC-SOS decomposition is very useful for difference-of-convex (DC) programming formulation of polynomial optimization problems. First, we introduce the cone of convex-sums-of-squares (CSOS) polynomials and discuss its relationship to the sums-of-squares (SOS) polynomials, the non-negative polynomials, and the SOS-convex polynomials. Then we propose the set of D-SOS and DC-SOS polynomials and prove that any polynomial can be formulated as D-SOS and DC-SOS. The problem of finding D-SOS and DC-SOS decompositions can be formulated as a semi-definite program and solved for any desired precision in polynomial time using interior point methods. Some algebraic properties of CSOS, D-SOS, and DC-SOS are established. Second, we focus on establishing several practical algorithms for exact D-SOS and DC-SOS polynomial decompositions without solving any SDP. The numerical performance of the proposed D-SOS and DC-SOS decomposition algorithms and their parallel versions, tested on a dataset of 1750 randomly generated polynomials, is reported.","PeriodicalId":49529,"journal":{"name":"SIAM Journal on Optimization","volume":null,"pages":null},"PeriodicalIF":3.1,"publicationDate":"2024-05-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141147262","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Vector Asymptotic Functions and Their Application to Multiobjective Optimization Problems","authors":"F. Flores-Bazán, Rubén López, C. Vera","doi":"10.1137/23m158098x","DOIUrl":"https://doi.org/10.1137/23m158098x","url":null,"abstract":"","PeriodicalId":49529,"journal":{"name":"SIAM Journal on Optimization","volume":null,"pages":null},"PeriodicalIF":3.1,"publicationDate":"2024-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141108736","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Extending the Rademacher Theorem to Set-Valued Maps","authors":"Aris Daniilidis, Marc Quincampoix","doi":"10.1137/22m1538831","DOIUrl":"https://doi.org/10.1137/22m1538831","url":null,"abstract":"","PeriodicalId":49529,"journal":{"name":"SIAM Journal on Optimization","volume":null,"pages":null},"PeriodicalIF":3.1,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141121729","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Roberto Andreani, Kelvin R. Couto, Orizon P. Ferreira, Gabriel Haeser
SIAM Journal on Optimization, Volume 34, Issue 2, Page 1799-1825, June 2024. Abstract. In the past several years, augmented Lagrangian methods have been successfully applied to several classes of nonconvex optimization problems, inspiring new developments in both theory and practice. In this paper we bring most of these recent developments from nonlinear programming to the context of optimization on Riemannian manifolds, including equality and inequality constraints. Many research have been conducted on optimization problems on manifolds, however only recently the treatment of the constrained case has been considered. In this paper we propose to bridge this gap with respect to the most recent developments in nonlinear programming. In particular, we formulate several well-known constraint qualifications from the Euclidean context which are sufficient for guaranteeing global convergence of augmented Lagrangian methods, without requiring boundedness of the set of Lagrange multipliers. Convergence of the dual sequence can also be assured under a weak constraint qualification. The theory presented is based on so-called sequential optimality conditions, which is a powerful tool used in this context. The paper can also be read with the Euclidean context in mind, serving as a review of the most relevant constraint qualifications and global convergence theory of state-of-the-art augmented Lagrangian methods for nonlinear programming.
{"title":"Constraint Qualifications and Strong Global Convergence Properties of an Augmented Lagrangian Method on Riemannian Manifolds","authors":"Roberto Andreani, Kelvin R. Couto, Orizon P. Ferreira, Gabriel Haeser","doi":"10.1137/23m1582382","DOIUrl":"https://doi.org/10.1137/23m1582382","url":null,"abstract":"SIAM Journal on Optimization, Volume 34, Issue 2, Page 1799-1825, June 2024. <br/> Abstract. In the past several years, augmented Lagrangian methods have been successfully applied to several classes of nonconvex optimization problems, inspiring new developments in both theory and practice. In this paper we bring most of these recent developments from nonlinear programming to the context of optimization on Riemannian manifolds, including equality and inequality constraints. Many research have been conducted on optimization problems on manifolds, however only recently the treatment of the constrained case has been considered. In this paper we propose to bridge this gap with respect to the most recent developments in nonlinear programming. In particular, we formulate several well-known constraint qualifications from the Euclidean context which are sufficient for guaranteeing global convergence of augmented Lagrangian methods, without requiring boundedness of the set of Lagrange multipliers. Convergence of the dual sequence can also be assured under a weak constraint qualification. The theory presented is based on so-called sequential optimality conditions, which is a powerful tool used in this context. The paper can also be read with the Euclidean context in mind, serving as a review of the most relevant constraint qualifications and global convergence theory of state-of-the-art augmented Lagrangian methods for nonlinear programming.","PeriodicalId":49529,"journal":{"name":"SIAM Journal on Optimization","volume":null,"pages":null},"PeriodicalIF":3.1,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141147261","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
E. Ruben van Beesten, Ward Romeijnders, David P. Morton
SIAM Journal on Optimization, Volume 34, Issue 2, Page 1755-1783, June 2024. Abstract. Inspired by its success for their continuous counterparts, the standard approach to deal with mixed-integer recourse (MIR) models under distributional uncertainty is to use distributionally robust optimization (DRO). We argue, however, that this modeling choice is not always justified since DRO techniques are generally computationally challenging when integer decision variables are involved. That is why we propose an alternative approach for dealing with distributional uncertainty for the special case of simple integer recourse (SIR) models, which is aimed at obtaining models with improved computational tractability. We show that such models can be obtained by pragmatically selecting the uncertainty set. Here, we consider uncertainty sets based on the Wasserstein distance and also on generalized moment conditions. We compare our approach with standard DRO both numerically and theoretically. An important side result of our analysis is the derivation of performance guarantees for convex approximations of SIR models. In contrast to the literature, these error bounds are not only valid for a continuous distribution but hold for any distribution.
{"title":"Pragmatic Distributionally Robust Optimization for Simple Integer Recourse Models","authors":"E. Ruben van Beesten, Ward Romeijnders, David P. Morton","doi":"10.1137/22m1523509","DOIUrl":"https://doi.org/10.1137/22m1523509","url":null,"abstract":"SIAM Journal on Optimization, Volume 34, Issue 2, Page 1755-1783, June 2024. <br/> Abstract. Inspired by its success for their continuous counterparts, the standard approach to deal with mixed-integer recourse (MIR) models under distributional uncertainty is to use distributionally robust optimization (DRO). We argue, however, that this modeling choice is not always justified since DRO techniques are generally computationally challenging when integer decision variables are involved. That is why we propose an alternative approach for dealing with distributional uncertainty for the special case of simple integer recourse (SIR) models, which is aimed at obtaining models with improved computational tractability. We show that such models can be obtained by pragmatically selecting the uncertainty set. Here, we consider uncertainty sets based on the Wasserstein distance and also on generalized moment conditions. We compare our approach with standard DRO both numerically and theoretically. An important side result of our analysis is the derivation of performance guarantees for convex approximations of SIR models. In contrast to the literature, these error bounds are not only valid for a continuous distribution but hold for any distribution.","PeriodicalId":49529,"journal":{"name":"SIAM Journal on Optimization","volume":null,"pages":null},"PeriodicalIF":3.1,"publicationDate":"2024-05-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141061877","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号