首页 > 最新文献

Computational Optimization and Applications最新文献

英文 中文
Dynamic stochastic projection method for multistage stochastic variational inequalities 多阶段随机变分不等式的动态随机投影法
IF 2.2 2区 数学 Q2 MATHEMATICS, APPLIED Pub Date : 2024-07-26 DOI: 10.1007/s10589-024-00594-4
Bin Zhou, Jie Jiang, Hailin Sun

Stochastic approximation (SA) type methods have been well studied for solving single-stage stochastic variational inequalities (SVIs). This paper proposes a dynamic stochastic projection method (DSPM) for solving multistage SVIs. In particular, we investigate an inexact single-stage SVI and present an inexact stochastic projection method (ISPM) for solving it. Then we give the DSPM to a three-stage SVI by applying the ISPM to each stage. We show that the DSPM can achieve an (mathcal {O}(frac{1}{epsilon ^2})) convergence rate regarding to the total number of required scenarios for the three-stage SVI. We also extend the DSPM to the multistage SVI when the number of stages is larger than three. The numerical experiments illustrate the effectiveness and efficiency of the DSPM.

对于求解单阶段随机变分不等式(SVI),随机逼近(SA)类型的方法已经得到了深入研究。本文提出了一种用于求解多阶段 SVI 的动态随机投影法 (DSPM)。我们特别研究了一个不精确的单阶段 SVI,并提出了一种用于求解该 SVI 的不精确随机投影法 (ISPM)。然后,我们通过对每个阶段应用 ISPM,将 DSPM 应用于三阶段 SVI。我们证明,对于三阶段 SVI 所需的场景总数,DSPM 可以达到 (mathcal {O}(frac{1}{epsilon ^2}))收敛率。当阶段数大于三个时,我们还将 DSPM 扩展到多阶段 SVI。数值实验说明了 DSPM 的有效性和效率。
{"title":"Dynamic stochastic projection method for multistage stochastic variational inequalities","authors":"Bin Zhou, Jie Jiang, Hailin Sun","doi":"10.1007/s10589-024-00594-4","DOIUrl":"https://doi.org/10.1007/s10589-024-00594-4","url":null,"abstract":"<p>Stochastic approximation (SA) type methods have been well studied for solving single-stage stochastic variational inequalities (SVIs). This paper proposes a dynamic stochastic projection method (DSPM) for solving multistage SVIs. In particular, we investigate an inexact single-stage SVI and present an inexact stochastic projection method (ISPM) for solving it. Then we give the DSPM to a three-stage SVI by applying the ISPM to each stage. We show that the DSPM can achieve an <span>(mathcal {O}(frac{1}{epsilon ^2}))</span> convergence rate regarding to the total number of required scenarios for the three-stage SVI. We also extend the DSPM to the multistage SVI when the number of stages is larger than three. The numerical experiments illustrate the effectiveness and efficiency of the DSPM.</p>","PeriodicalId":55227,"journal":{"name":"Computational Optimization and Applications","volume":"7 1","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-07-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141780193","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Extragradient method with feasible inexact projection to variational inequality problem 针对变分不等式问题的可行不精确投影外梯度法
IF 2.2 2区 数学 Q2 MATHEMATICS, APPLIED Pub Date : 2024-07-26 DOI: 10.1007/s10589-024-00592-6
R. Díaz Millán, O. P. Ferreira, J. Ugon

The variational inequality problem in finite-dimensional Euclidean space is addressed in this paper, and two inexact variants of the extragradient method are proposed to solve it. Instead of computing exact projections on the constraint set, as in previous versions extragradient method, the proposed methods compute feasible inexact projections on the constraint set using a relative error criterion. The first version of the proposed method provided is a counterpart to the classic form of the extragradient method with constant steps. In order to establish its convergence we need to assume that the operator is pseudo-monotone and Lipschitz continuous, as in the standard approach. For the second version, instead of a fixed step size, the method presented finds a suitable step size in each iteration by performing a line search. Like the classical extragradient method, the proposed method does just two projections into the feasible set in each iteration. A full convergence analysis is provided, with no Lipschitz continuity assumption of the operator defining the variational inequality problem.

本文探讨了有限维欧几里得空间中的变分不等式问题,并提出了两种外梯度法的非精确变体来解决这一问题。所提出的方法不像以前的外梯度法那样计算约束集上的精确投影,而是利用相对误差准则计算约束集上可行的非精确投影。所提出方法的第一个版本与经典形式的外梯度法相对应,具有恒定步长。为了确定其收敛性,我们需要假设算子是伪单调和 Lipschitz 连续的,就像标准方法一样。对于第二个版本,所介绍的方法不是固定步长,而是通过线性搜索在每次迭代中找到合适的步长。与经典的外梯度法一样,所提出的方法在每次迭代中只对可行集进行两次投影。在不对定义变分不等式问题的算子进行 Lipschitz 连续性假设的情况下,提供了完整的收敛性分析。
{"title":"Extragradient method with feasible inexact projection to variational inequality problem","authors":"R. Díaz Millán, O. P. Ferreira, J. Ugon","doi":"10.1007/s10589-024-00592-6","DOIUrl":"https://doi.org/10.1007/s10589-024-00592-6","url":null,"abstract":"<p>The variational inequality problem in finite-dimensional Euclidean space is addressed in this paper, and two inexact variants of the extragradient method are proposed to solve it. Instead of computing exact projections on the constraint set, as in previous versions extragradient method, the proposed methods compute feasible inexact projections on the constraint set using a relative error criterion. The first version of the proposed method provided is a counterpart to the classic form of the extragradient method with constant steps. In order to establish its convergence we need to assume that the operator is pseudo-monotone and Lipschitz continuous, as in the standard approach. For the second version, instead of a fixed step size, the method presented finds a suitable step size in each iteration by performing a line search. Like the classical extragradient method, the proposed method does just two projections into the feasible set in each iteration. A full convergence analysis is provided, with no Lipschitz continuity assumption of the operator defining the variational inequality problem.</p>","PeriodicalId":55227,"journal":{"name":"Computational Optimization and Applications","volume":"42 1","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-07-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141780195","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Handling of constraints in multiobjective blackbox optimization 多目标黑箱优化中的约束处理
IF 2.2 2区 数学 Q2 MATHEMATICS, APPLIED Pub Date : 2024-07-16 DOI: 10.1007/s10589-024-00588-2
Jean Bigeon, Sébastien Le Digabel, Ludovic Salomon

This work proposes the integration of two new constraint-handling approaches into the blackbox constrained multiobjective optimization algorithm DMulti-MADS, an extension of the Mesh Adaptive Direct Search (MADS) algorithm for single-objective constrained optimization. The constraints are aggregated into a single constraint violation function which is used either in a two-phase approach, where the search for a feasible point is prioritized if not available before improving the current solution set, or in a progressive barrier approach, where any trial point whose constraint violation function values are above a threshold are rejected. This threshold is progressively decreased along the iterations. As in the single-objective case, it is proved that these two variants generate feasible and/or infeasible sequences which converge either in the feasible case to a set of local Pareto optimal points or in the infeasible case to Clarke stationary points according to the constraint violation function. Computational experiments show that these two approaches are competitive with other state-of-the-art algorithms.

本研究提出将两种新的约束处理方法整合到黑盒约束多目标优化算法 DMulti-MADS 中,DMulti-MADS 是用于单目标约束优化的网格自适应直接搜索(MADS)算法的扩展。约束条件被汇总为一个单一的约束违规函数,该函数可用于两阶段方法,即在改进当前解集之前,如果没有可行点,则优先搜索可行点;或用于渐进障碍方法,即拒绝约束违规函数值高于阈值的任何试验点。在迭代过程中,阈值会逐渐降低。正如在单目标情况下一样,实验证明这两种方法都能产生可行和/或不可行序列,在可行情况下,这些序列会收敛到一组局部帕累托最优点,在不可行情况下,会根据违反约束函数收敛到克拉克静止点。计算实验表明,这两种方法与其他最先进的算法相比具有竞争力。
{"title":"Handling of constraints in multiobjective blackbox optimization","authors":"Jean Bigeon, Sébastien Le Digabel, Ludovic Salomon","doi":"10.1007/s10589-024-00588-2","DOIUrl":"https://doi.org/10.1007/s10589-024-00588-2","url":null,"abstract":"<p>This work proposes the integration of two new constraint-handling approaches into the blackbox constrained multiobjective optimization algorithm DMulti-MADS, an extension of the Mesh Adaptive Direct Search (MADS) algorithm for single-objective constrained optimization. The constraints are aggregated into a single constraint violation function which is used either in a two-phase approach, where the search for a feasible point is prioritized if not available before improving the current solution set, or in a progressive barrier approach, where any trial point whose constraint violation function values are above a threshold are rejected. This threshold is progressively decreased along the iterations. As in the single-objective case, it is proved that these two variants generate feasible and/or infeasible sequences which converge either in the feasible case to a set of local Pareto optimal points or in the infeasible case to Clarke stationary points according to the constraint violation function. Computational experiments show that these two approaches are competitive with other state-of-the-art algorithms.</p>","PeriodicalId":55227,"journal":{"name":"Computational Optimization and Applications","volume":"38 1","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-07-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141721903","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Eigenvalue programming beyond matrices 超越矩阵的特征值编程
IF 2.2 2区 数学 Q2 MATHEMATICS, APPLIED Pub Date : 2024-07-10 DOI: 10.1007/s10589-024-00591-7
Masaru Ito, Bruno F. Lourenço

In this paper we analyze and solve eigenvalue programs, which consist of the task of minimizing a function subject to constraints on the “eigenvalues” of the decision variable. Here, by making use of the FTvN systems framework introduced by Gowda, we interpret “eigenvalues” in a broad fashion going beyond the usual eigenvalues of matrices. This allows us to shed new light on classical problems such as inverse eigenvalue problems and also leads to new applications. In particular, after analyzing and developing a simple projected gradient algorithm for general eigenvalue programs, we show that eigenvalue programs can be used to express what we call vanishing quadratic constraints. A vanishing quadratic constraint requires that a given system of convex quadratic inequalities be satisfied and at least a certain number of those inequalities must be tight. As a particular case, this includes the problem of finding a point x in the intersection of m ellipsoids in such a way that x is also in the boundary of at least (ell ) of the ellipsoids, for some fixed (ell > 0). At the end, we also present some numerical experiments.

在本文中,我们分析并求解了特征值程序,该程序包括在决策变量 "特征值 "的约束下最小化函数的任务。在这里,通过利用高达提出的 FTvN 系统框架,我们对 "特征值 "进行了广义的解释,超越了通常的矩阵特征值。这使我们能够为逆特征值问题等经典问题带来新的启示,同时也带来了新的应用。特别是,在分析和开发了一般特征值程序的简单投影梯度算法后,我们证明特征值程序可以用来表达我们称之为消失二次约束的问题。消失二次约束要求满足给定的凸二次不等式系统,并且其中至少有一定数量的不等式必须是严密的。作为一种特殊情况,这包括在 m 个椭圆的交点上找到一个点 x,使得 x 在某个固定的 (ell > 0) 的情况下,也在至少 (ell ) 个椭圆的边界上。最后,我们还介绍了一些数值实验。
{"title":"Eigenvalue programming beyond matrices","authors":"Masaru Ito, Bruno F. Lourenço","doi":"10.1007/s10589-024-00591-7","DOIUrl":"https://doi.org/10.1007/s10589-024-00591-7","url":null,"abstract":"<p>In this paper we analyze and solve eigenvalue programs, which consist of the task of minimizing a function subject to constraints on the “eigenvalues” of the decision variable. Here, by making use of the FTvN systems framework introduced by Gowda, we interpret “eigenvalues” in a broad fashion going beyond the usual eigenvalues of matrices. This allows us to shed new light on classical problems such as inverse eigenvalue problems and also leads to new applications. In particular, after analyzing and developing a simple projected gradient algorithm for general eigenvalue programs, we show that eigenvalue programs can be used to express what we call <i>vanishing quadratic constraints</i>. A vanishing quadratic constraint requires that a given system of convex quadratic inequalities be satisfied and at least a certain number of those inequalities must be tight. As a particular case, this includes the problem of finding a point <i>x</i> in the intersection of <i>m</i> ellipsoids in such a way that <i>x</i> is also in the boundary of at least <span>(ell )</span> of the ellipsoids, for some fixed <span>(ell &gt; 0)</span>. At the end, we also present some numerical experiments.</p>","PeriodicalId":55227,"journal":{"name":"Computational Optimization and Applications","volume":"31 1","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141587285","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Q-fully quadratic modeling and its application in a random subspace derivative-free method 全二次方建模及其在随机子空间无导数方法中的应用
IF 2.2 2区 数学 Q2 MATHEMATICS, APPLIED Pub Date : 2024-06-20 DOI: 10.1007/s10589-024-00590-8
Yiwen Chen, Warren Hare, Amy Wiebe

Model-based derivative-free optimization (DFO) methods are an important class of DFO methods that are known to struggle with solving high-dimensional optimization problems. Recent research has shown that incorporating random subspaces into model-based DFO methods has the potential to improve their performance on high-dimensional problems. However, most of the current theoretical and practical results are based on linear approximation models due to the complexity of quadratic approximation models. This paper proposes a random subspace trust-region algorithm based on quadratic approximations. Unlike most of its precursors, this algorithm does not require any special form of objective function. We study the geometry of sample sets, the error bounds for approximations, and the quality of subspaces. In particular, we provide a technique to construct Q-fully quadratic models, which is easy to analyze and implement. We present an almost-sure global convergence result of our algorithm and give an upper bound on the expected number of iterations to find a sufficiently small gradient. We also develop numerical experiments to compare the performance of our algorithm using both linear and quadratic approximation models. The numerical results demonstrate the strengths and weaknesses of using quadratic approximations.

基于模型的无导数优化(DFO)方法是一类重要的无导数优化方法,众所周知,这类方法在解决高维优化问题时比较吃力。最近的研究表明,将随机子空间纳入基于模型的无导数优化方法有可能提高其在高维问题上的性能。然而,由于二次近似模型的复杂性,目前大多数理论和实践成果都是基于线性近似模型的。本文提出了一种基于二次逼近的随机子空间信任区域算法。与大多数前辈算法不同,该算法不需要任何特殊形式的目标函数。我们研究了样本集的几何形状、近似的误差边界以及子空间的质量。特别是,我们提供了一种构建 Q 全二次模型的技术,这种技术易于分析和实现。我们提出了算法几乎可以确定的全局收敛结果,并给出了找到足够小梯度的预期迭代次数上限。我们还进行了数值实验,使用线性和二次逼近模型比较了我们算法的性能。数值结果表明了使用二次逼近的优缺点。
{"title":"Q-fully quadratic modeling and its application in a random subspace derivative-free method","authors":"Yiwen Chen, Warren Hare, Amy Wiebe","doi":"10.1007/s10589-024-00590-8","DOIUrl":"https://doi.org/10.1007/s10589-024-00590-8","url":null,"abstract":"<p>Model-based derivative-free optimization (DFO) methods are an important class of DFO methods that are known to struggle with solving high-dimensional optimization problems. Recent research has shown that incorporating random subspaces into model-based DFO methods has the potential to improve their performance on high-dimensional problems. However, most of the current theoretical and practical results are based on linear approximation models due to the complexity of quadratic approximation models. This paper proposes a random subspace trust-region algorithm based on quadratic approximations. Unlike most of its precursors, this algorithm does not require any special form of objective function. We study the geometry of sample sets, the error bounds for approximations, and the quality of subspaces. In particular, we provide a technique to construct <i>Q</i>-fully quadratic models, which is easy to analyze and implement. We present an almost-sure global convergence result of our algorithm and give an upper bound on the expected number of iterations to find a sufficiently small gradient. We also develop numerical experiments to compare the performance of our algorithm using both linear and quadratic approximation models. The numerical results demonstrate the strengths and weaknesses of using quadratic approximations.</p>","PeriodicalId":55227,"journal":{"name":"Computational Optimization and Applications","volume":"17 1","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141505104","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A nonsmooth primal-dual method with interwoven PDE constraint solver 带有交织 PDE 约束求解器的非平滑原始二元法
IF 2.2 2区 数学 Q2 MATHEMATICS, APPLIED Pub Date : 2024-06-08 DOI: 10.1007/s10589-024-00587-3
Bjørn Jensen, Tuomo Valkonen

We introduce an efficient first-order primal-dual method for the solution of nonsmooth PDE-constrained optimization problems. We achieve this efficiency through not solving the PDE or its linearisation on each iteration of the optimization method. Instead, we run the method interwoven with a simple conventional linear system solver (Jacobi, Gauss–Seidel, conjugate gradients), always taking only one step of the linear system solver for each step of the optimization method. The control parameter is updated on each iteration as determined by the optimization method. We prove linear convergence under a second-order growth condition, and numerically demonstrate the performance on a variety of PDEs related to inverse problems involving boundary measurements.

我们介绍了一种高效的一阶初等二元方法,用于解决非光滑 PDE 受限优化问题。我们不在优化方法的每次迭代中求解 PDE 或其线性化,从而实现了这种效率。相反,我们将该方法与简单的传统线性系统求解器(雅可比、高斯-赛德尔、共轭梯度)交织运行,优化方法的每一步都只用线性系统求解器的一步。每次迭代都会根据优化方法更新控制参数。我们证明了在二阶增长条件下的线性收敛性,并对涉及边界测量的逆问题的各种 PDE 进行了数值演示。
{"title":"A nonsmooth primal-dual method with interwoven PDE constraint solver","authors":"Bjørn Jensen, Tuomo Valkonen","doi":"10.1007/s10589-024-00587-3","DOIUrl":"https://doi.org/10.1007/s10589-024-00587-3","url":null,"abstract":"<p>We introduce an efficient first-order primal-dual method for the solution of nonsmooth PDE-constrained optimization problems. We achieve this efficiency through <i>not</i> solving the PDE or its linearisation on each iteration of the optimization method. Instead, we run the method interwoven with a simple conventional linear system solver (Jacobi, Gauss–Seidel, conjugate gradients), always taking only <i>one step</i> of the linear system solver for each step of the optimization method. The control parameter is updated on each iteration as determined by the optimization method. We prove linear convergence under a second-order growth condition, and numerically demonstrate the performance on a variety of PDEs related to inverse problems involving boundary measurements.</p>","PeriodicalId":55227,"journal":{"name":"Computational Optimization and Applications","volume":"45 1","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-06-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141529143","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Stochastic Steffensen method 随机斯蒂芬森法
IF 2.2 2区 数学 Q2 MATHEMATICS, APPLIED Pub Date : 2024-06-07 DOI: 10.1007/s10589-024-00583-7
Minda Zhao, Zehua Lai, Lek-Heng Lim

Is it possible for a first-order method, i.e., only first derivatives allowed, to be quadratically convergent? For univariate loss functions, the answer is yes—the Steffensen method avoids second derivatives and is still quadratically convergent like Newton method. By incorporating a specific step size we can even push its convergence order beyond quadratic to (1+sqrt{2} approx 2.414). While such high convergence orders are a pointless overkill for a deterministic algorithm, they become rewarding when the algorithm is randomized for problems of massive sizes, as randomization invariably compromises convergence speed. We will introduce two adaptive learning rates inspired by the Steffensen method, intended for use in a stochastic optimization setting and requires no hyperparameter tuning aside from batch size. Extensive experiments show that they compare favorably with several existing first-order methods. When restricted to a quadratic objective, our stochastic Steffensen methods reduce to randomized Kaczmarz method—note that this is not true for SGD or SLBFGS—and thus we may also view our methods as a generalization of randomized Kaczmarz to arbitrary objectives.

一阶方法(即只允许一阶导数)有可能二次收敛吗?对于单变量损失函数,答案是肯定的--Steffensen 方法避免了二阶导数,仍然像牛顿方法一样具有二次收敛性。通过加入特定的步长,我们甚至可以把它的收敛阶数从二次收敛提高到(1+sqrt{2} 约 2.414)。对于一个确定性算法来说,如此高的收敛阶数是毫无意义的矫枉过正,但当算法被随机化以处理大规模问题时,它们就变得有价值了,因为随机化无形中会影响收敛速度。我们将介绍两种受 Steffensen 方法启发的自适应学习率,它们适用于随机优化环境,除了批量大小外无需调整超参数。广泛的实验表明,这两种方法与现有的几种一阶方法相比毫不逊色。当局限于二次目标时,我们的随机 Steffensen 方法会简化为随机 Kaczmarz 方法--请注意,SGD 或 SLBFGS 并非如此--因此,我们也可以将我们的方法视为随机 Kaczmarz 对任意目标的泛化。
{"title":"Stochastic Steffensen method","authors":"Minda Zhao, Zehua Lai, Lek-Heng Lim","doi":"10.1007/s10589-024-00583-7","DOIUrl":"https://doi.org/10.1007/s10589-024-00583-7","url":null,"abstract":"<p>Is it possible for a first-order method, i.e., only first derivatives allowed, to be quadratically convergent? For univariate loss functions, the answer is yes—the <i>Steffensen method</i> avoids second derivatives and is still quadratically convergent like Newton method. By incorporating a specific step size we can even push its convergence order beyond quadratic to <span>(1+sqrt{2} approx 2.414)</span>. While such high convergence orders are a pointless overkill for a deterministic algorithm, they become rewarding when the algorithm is randomized for problems of massive sizes, as randomization invariably compromises convergence speed. We will introduce two adaptive learning rates inspired by the Steffensen method, intended for use in a stochastic optimization setting and requires no hyperparameter tuning aside from batch size. Extensive experiments show that they compare favorably with several existing first-order methods. When restricted to a quadratic objective, our stochastic Steffensen methods reduce to randomized Kaczmarz method—note that this is not true for SGD or SLBFGS—and thus we may also view our methods as a generalization of randomized Kaczmarz to arbitrary objectives.</p>","PeriodicalId":55227,"journal":{"name":"Computational Optimization and Applications","volume":"8 1","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-06-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141548136","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Polynomial worst-case iteration complexity of quasi-Newton primal-dual interior point algorithms for linear programming 线性规划准牛顿原始双内点算法的多项式最坏迭代复杂度
IF 2.2 2区 数学 Q2 MATHEMATICS, APPLIED Pub Date : 2024-06-07 DOI: 10.1007/s10589-024-00584-6
Jacek Gondzio, Francisco N. C. Sobral

Quasi-Newton methods are well known techniques for large-scale numerical optimization. They use an approximation of the Hessian in optimization problems or the Jacobian in system of nonlinear equations. In the Interior Point context, quasi-Newton algorithms compute low-rank updates of the matrix associated with the Newton systems, instead of computing it from scratch at every iteration. In this work, we show that a simplified quasi-Newton primal-dual interior point algorithm for linear programming, which alternates between Newton and quasi-Newton iterations, enjoys polynomial worst-case iteration complexity. Feasible and infeasible cases of the algorithm are considered and the most common neighborhoods of the central path are analyzed. To the best of our knowledge, this is the first attempt to deliver polynomial worst-case iteration complexity bounds for these methods. Unsurprisingly, the worst-case complexity results obtained when quasi-Newton directions are used are worse than their counterparts when Newton directions are employed. However, quasi-Newton updates are very attractive for large-scale optimization problems where the cost of factorizing the matrices is much higher than the cost of solving linear systems.

准牛顿方法是众所周知的大规模数值优化技术。它们在优化问题中使用 Hessian 近似值,在非线性方程组中使用 Jacobian 近似值。在内点法的背景下,准牛顿算法计算与牛顿系统相关的矩阵的低秩更新,而不是在每次迭代时从头开始计算。在这项研究中,我们展示了一种简化的线性规划准牛顿原始双内点算法,它交替进行牛顿和准牛顿迭代,在最坏情况下具有多项式迭代复杂度。该算法考虑了可行和不可行的情况,并分析了中心路径最常见的邻域。据我们所知,这是首次尝试为这些方法提供多项式最坏情况迭代复杂度边界。不出所料,使用准牛顿方向时获得的最坏情况复杂度结果比使用牛顿方向时的结果要差。然而,准牛顿更新对大规模优化问题非常有吸引力,因为在这些问题中,矩阵因式分解的成本远远高于求解线性系统的成本。
{"title":"Polynomial worst-case iteration complexity of quasi-Newton primal-dual interior point algorithms for linear programming","authors":"Jacek Gondzio, Francisco N. C. Sobral","doi":"10.1007/s10589-024-00584-6","DOIUrl":"https://doi.org/10.1007/s10589-024-00584-6","url":null,"abstract":"<p>Quasi-Newton methods are well known techniques for large-scale numerical optimization. They use an approximation of the Hessian in optimization problems or the Jacobian in system of nonlinear equations. In the Interior Point context, quasi-Newton algorithms compute low-rank updates of the matrix associated with the Newton systems, instead of computing it from scratch at every iteration. In this work, we show that a simplified quasi-Newton primal-dual interior point algorithm for linear programming, which alternates between Newton and quasi-Newton iterations, enjoys polynomial worst-case iteration complexity. Feasible and infeasible cases of the algorithm are considered and the most common neighborhoods of the central path are analyzed. To the best of our knowledge, this is the first attempt to deliver polynomial worst-case iteration complexity bounds for these methods. Unsurprisingly, the worst-case complexity results obtained when quasi-Newton directions are used are worse than their counterparts when Newton directions are employed. However, quasi-Newton updates are very attractive for large-scale optimization problems where the cost of factorizing the matrices is much higher than the cost of solving linear systems.</p>","PeriodicalId":55227,"journal":{"name":"Computational Optimization and Applications","volume":"42 1","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-06-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141548135","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Projection free methods on product domains 乘积域上的无投影方法
IF 2.2 2区 数学 Q2 MATHEMATICS, APPLIED Pub Date : 2024-06-04 DOI: 10.1007/s10589-024-00585-5
Immanuel Bomze, Francesco Rinaldi, Damiano Zeffiro

Projection-free block-coordinate methods avoid high computational cost per iteration, and at the same time exploit the particular problem structure of product domains. Frank–Wolfe-like approaches rank among the most popular ones of this type. However, as observed in the literature, there was a gap between the classical Frank–Wolfe theory and the block-coordinate case, with no guarantees of linear convergence rates even for strongly convex objectives in the latter. Moreover, most of previous research concentrated on convex objectives. This study now deals also with the non-convex case and reduces above-mentioned theory gap, in combining a new, fully developed convergence theory with novel active set identification results which ensure that inherent sparsity of solutions can be exploited in an efficient way. Preliminary numerical experiments seem to justify our approach and also show promising results for obtaining global solutions in the non-convex case.

无投影块坐标方法可以避免每次迭代的高计算成本,同时还能利用积域的特殊问题结构。类似弗兰克-沃尔夫的方法是这类方法中最流行的一种。然而,从文献中可以看出,经典的弗兰克-沃尔夫理论与块坐标情况之间存在差距,即使是后者中的强凸目标,也无法保证线性收敛率。此外,以前的研究大多集中于凸目标。现在,这项研究也涉及非凸情况,并将新的、全面发展的收敛理论与新的主动集识别结果相结合,确保以有效的方式利用解的固有稀疏性,从而缩小了上述理论差距。初步的数值实验似乎证明了我们的方法是正确的,同时也显示了在非凸情况下获得全局解的可喜成果。
{"title":"Projection free methods on product domains","authors":"Immanuel Bomze, Francesco Rinaldi, Damiano Zeffiro","doi":"10.1007/s10589-024-00585-5","DOIUrl":"https://doi.org/10.1007/s10589-024-00585-5","url":null,"abstract":"<p>Projection-free block-coordinate methods avoid high computational cost per iteration, and at the same time exploit the particular problem structure of product domains. Frank–Wolfe-like approaches rank among the most popular ones of this type. However, as observed in the literature, there was a gap between the classical Frank–Wolfe theory and the block-coordinate case, with no guarantees of linear convergence rates even for strongly convex objectives in the latter. Moreover, most of previous research concentrated on convex objectives. This study now deals also with the non-convex case and reduces above-mentioned theory gap, in combining a new, fully developed convergence theory with novel active set identification results which ensure that inherent sparsity of solutions can be exploited in an efficient way. Preliminary numerical experiments seem to justify our approach and also show promising results for obtaining global solutions in the non-convex case.</p>","PeriodicalId":55227,"journal":{"name":"Computational Optimization and Applications","volume":"128 1","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141258500","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
T-semidefinite programming relaxation with third-order tensors for constrained polynomial optimization 用三阶张量对受约束多项式优化进行 T-半无限编程放松
IF 2.2 2区 数学 Q2 MATHEMATICS, APPLIED Pub Date : 2024-06-02 DOI: 10.1007/s10589-024-00582-8
Hiroki Marumo, Sunyoung Kim, Makoto Yamashita

We study T-semidefinite programming (SDP) relaxation for constrained polynomial optimization problems (POPs). T-SDP relaxation for unconstrained POPs was introduced by Zheng et al. (JGO 84:415–440, 2022). In this work, we propose a T-SDP relaxation for POPs with polynomial inequality constraints and show that the resulting T-SDP relaxation formulated with third-order tensors can be transformed into the standard SDP relaxation with block-diagonal structures. The convergence of the T-SDP relaxation to the optimal value of a given constrained POP is established under moderate assumptions as the relaxation level increases. Additionally, the feasibility and optimality of the T-SDP relaxation are discussed. Numerical results illustrate that the proposed T-SDP relaxation enhances numerical efficiency.

我们研究受约束多项式优化问题(POPs)的T-半有限编程(SDP)松弛。Zheng 等(JGO 84:415-440, 2022)提出了针对无约束 POP 的 T-SDP 松弛法。在这项研究中,我们提出了一种针对多项式不等式约束的 POP 的 T-SDP 松弛,并证明了用三阶张量表述的 T-SDP 松弛可以转化为具有块对角结构的标准 SDP 松弛。随着松弛程度的增加,在适度假设条件下,T-SDP 松弛可以收敛到给定约束 POP 的最优值。此外,还讨论了 T-SDP 松弛的可行性和最优性。数值结果表明,建议的 T-SDP 松弛提高了数值效率。
{"title":"T-semidefinite programming relaxation with third-order tensors for constrained polynomial optimization","authors":"Hiroki Marumo, Sunyoung Kim, Makoto Yamashita","doi":"10.1007/s10589-024-00582-8","DOIUrl":"https://doi.org/10.1007/s10589-024-00582-8","url":null,"abstract":"<p>We study T-semidefinite programming (SDP) relaxation for constrained polynomial optimization problems (POPs). T-SDP relaxation for unconstrained POPs was introduced by Zheng et al. (JGO 84:415–440, 2022). In this work, we propose a T-SDP relaxation for POPs with polynomial inequality constraints and show that the resulting T-SDP relaxation formulated with third-order tensors can be transformed into the standard SDP relaxation with block-diagonal structures. The convergence of the T-SDP relaxation to the optimal value of a given constrained POP is established under moderate assumptions as the relaxation level increases. Additionally, the feasibility and optimality of the T-SDP relaxation are discussed. Numerical results illustrate that the proposed T-SDP relaxation enhances numerical efficiency.</p>","PeriodicalId":55227,"journal":{"name":"Computational Optimization and Applications","volume":"1 1","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141195344","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
Computational Optimization and Applications
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1