Pub Date : 2024-04-09DOI: 10.1007/s10107-024-02082-3
Sebastian Lämmel, Vladimir Shikhman
We extend the convergence analysis of the Scholtes-type regularization method for cardinality-constrained optimization problems. Its behavior is clarified in the vicinity of saddle points, and not just of minimizers as it has been done in the literature before. This becomes possible by using as an intermediate step the recently introduced regularized continuous reformulation of a cardinality-constrained optimization problem. We show that the Scholtes-type regularization method is well-defined locally around a nondegenerate T-stationary point of this regularized continuous reformulation. Moreover, the nondegenerate Karush–Kuhn–Tucker points of the corresponding Scholtes-type regularization converge to a T-stationary point having the same index, i.e. its topological type persists. As consequence, we conclude that the global structure of the Scholtes-type regularization essentially coincides with that of CCOP.
{"title":"Extended convergence analysis of the Scholtes-type regularization for cardinality-constrained optimization problems","authors":"Sebastian Lämmel, Vladimir Shikhman","doi":"10.1007/s10107-024-02082-3","DOIUrl":"https://doi.org/10.1007/s10107-024-02082-3","url":null,"abstract":"<p>We extend the convergence analysis of the Scholtes-type regularization method for cardinality-constrained optimization problems. Its behavior is clarified in the vicinity of saddle points, and not just of minimizers as it has been done in the literature before. This becomes possible by using as an intermediate step the recently introduced regularized continuous reformulation of a cardinality-constrained optimization problem. We show that the Scholtes-type regularization method is well-defined locally around a nondegenerate T-stationary point of this regularized continuous reformulation. Moreover, the nondegenerate Karush–Kuhn–Tucker points of the corresponding Scholtes-type regularization converge to a T-stationary point having the same index, i.e. its topological type persists. As consequence, we conclude that the global structure of the Scholtes-type regularization essentially coincides with that of CCOP.</p>","PeriodicalId":18297,"journal":{"name":"Mathematical Programming","volume":"48 1","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-04-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140885824","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-06DOI: 10.1007/s10107-024-02080-5
Gonzalo Muñoz, Joseph Paat, Álinson S. Xavier
A branch-and-bound (BB) tree certifies a dual bound on the value of an integer program. In this work, we introduce the tree compression problem (TCP): Given a BB treeTthat certifies a dual bound, can we obtain a smaller tree with the same (or stronger) bound by either (1) applying a different disjunction at some node inTor (2) removing leaves fromT? We believe such post-hoc analysis of BB trees may assist in identifying helpful general disjunctions in BB algorithms. We initiate our study by considering computational complexity and limitations of TCP. We then conduct experiments to evaluate the compressibility of realistic branch-and-bound trees generated by commonly-used branching strategies, using both an exact and a heuristic compression algorithm.
分支约束(BB)树证明了整数程序值的双重约束。在这项工作中,我们引入了树压缩问题(TCP):给定一棵证明了对偶约束的分支约束树 T,我们能否通过(1)在 T 中的某个节点应用不同的析取或(2)从 T 中删除叶子,得到一棵具有相同(或更强)约束的更小的树?我们相信,这种对 BB 树的事后分析可能有助于识别 BB 算法中有用的通用析取。我们的研究首先考虑了 TCP 的计算复杂性和局限性。然后,我们使用精确压缩算法和启发式压缩算法进行实验,以评估由常用分支策略生成的现实分支约束树的可压缩性。
{"title":"Compressing branch-and-bound trees","authors":"Gonzalo Muñoz, Joseph Paat, Álinson S. Xavier","doi":"10.1007/s10107-024-02080-5","DOIUrl":"https://doi.org/10.1007/s10107-024-02080-5","url":null,"abstract":"<p>A branch-and-bound (BB) tree certifies a dual bound on the value of an integer program. In this work, we introduce the tree compression problem (TCP): <i>Given a BB tree</i> <i>T</i> <i>that certifies a dual bound, can we obtain a smaller tree with the same (or stronger) bound by either (1) applying a different disjunction at some node in</i> <i>T</i> <i>or (2) removing leaves from</i> <i>T</i>? We believe such post-hoc analysis of BB trees may assist in identifying helpful general disjunctions in BB algorithms. We initiate our study by considering computational complexity and limitations of TCP. We then conduct experiments to evaluate the compressibility of realistic branch-and-bound trees generated by commonly-used branching strategies, using both an exact and a heuristic compression algorithm.</p>","PeriodicalId":18297,"journal":{"name":"Mathematical Programming","volume":"32 1","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-04-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140595792","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-04DOI: 10.1007/s10107-024-02081-4
Alessandro Rudi, Ulysse Marteau-Ferey, Francis Bach
We consider the global minimization of smooth functions based solely on function evaluations. Algorithms that achieve the optimal number of function evaluations for a given precision level typically rely on explicitly constructing an approximation of the function which is then minimized with algorithms that have exponential running-time complexity. In this paper, we consider an approach that jointly models the function to approximate and finds a global minimum. This is done by using infinite sums of square smooth functions and has strong links with polynomial sum-of-squares hierarchies. Leveraging recent representation properties of reproducing kernel Hilbert spaces, the infinite-dimensional optimization problem can be solved by subsampling in time polynomial in the number of function evaluations, and with theoretical guarantees on the obtained minimum. Given n samples, the computational cost is (O(n^{3.5})) in time, (O(n^2)) in space, and we achieve a convergence rate to the global optimum that is (O(n^{-m/d + 1/2 + 3/d})) where m is the degree of differentiability of the function and d the number of dimensions. The rate is nearly optimal in the case of Sobolev functions and more generally makes the proposed method particularly suitable for functions with many derivatives. Indeed, when m is in the order of d, the convergence rate to the global optimum does not suffer from the curse of dimensionality, which affects only the worst-case constants (that we track explicitly through the paper).
我们只考虑基于函数求值的平滑函数全局最小化问题。在给定精度水平下,实现最佳函数求值次数的算法通常依赖于显式构建函数近似值,然后用运行时间复杂度呈指数级的算法将其最小化。在本文中,我们考虑了一种方法,即联合建立函数近似模型并找到全局最小值。这种方法通过使用平方平滑函数的无限和来实现,并与多项式平方和层次结构有着密切联系。利用重现核希尔伯特空间的最新表示特性,可以通过子采样在函数求值次数为多项式的时间内求解无穷维优化问题,并从理论上保证求得最小值。给定 n 个样本,计算成本在时间上是(O(n^{3.5})),在空间上是(O(n^2)),我们达到全局最优的收敛速率是(O(n^{-m/d + 1/2 + 3/d})) 其中 m 是函数的可微分程度,d 是维数。在 Sobolev 函数的情况下,这个比率几乎是最优的,而且在更广泛的情况下,所提出的方法特别适用于具有许多导数的函数。事实上,当 m 在 d 的数量级时,向全局最优的收敛率不会受到维数诅咒的影响,维数诅咒只影响最坏情况下的常数(我们在论文中明确跟踪了这些常数)。
{"title":"Finding global minima via kernel approximations","authors":"Alessandro Rudi, Ulysse Marteau-Ferey, Francis Bach","doi":"10.1007/s10107-024-02081-4","DOIUrl":"https://doi.org/10.1007/s10107-024-02081-4","url":null,"abstract":"<p>We consider the global minimization of smooth functions based solely on function evaluations. Algorithms that achieve the optimal number of function evaluations for a given precision level typically rely on explicitly constructing an approximation of the function which is then minimized with algorithms that have exponential running-time complexity. In this paper, we consider an approach that jointly models the function to approximate and finds a global minimum. This is done by using infinite sums of square smooth functions and has strong links with polynomial sum-of-squares hierarchies. Leveraging recent representation properties of reproducing kernel Hilbert spaces, the infinite-dimensional optimization problem can be solved by subsampling in time polynomial in the number of function evaluations, and with theoretical guarantees on the obtained minimum. Given <i>n</i> samples, the computational cost is <span>(O(n^{3.5}))</span> in time, <span>(O(n^2))</span> in space, and we achieve a convergence rate to the global optimum that is <span>(O(n^{-m/d + 1/2 + 3/d}))</span> where <i>m</i> is the degree of differentiability of the function and <i>d</i> the number of dimensions. The rate is nearly optimal in the case of Sobolev functions and more generally makes the proposed method particularly suitable for functions with many derivatives. Indeed, when <i>m</i> is in the order of <i>d</i>, the convergence rate to the global optimum does not suffer from the curse of dimensionality, which affects only the worst-case constants (that we track explicitly through the paper).\u0000</p>","PeriodicalId":18297,"journal":{"name":"Mathematical Programming","volume":"2 1","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-04-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140595779","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-02DOI: 10.1007/s10107-024-02083-2
Abstract
Risk measures are commonly used to capture the risk preferences of decision-makers (DMs). The decisions of DMs can be nudged or manipulated when their risk preferences are influenced by factors such as the availability of information about the uncertainties. This work proposes a Stackelberg risk preference design (STRIPE) problem to capture a designer’s incentive to influence DMs’ risk preferences. STRIPE consists of two levels. In the lower level, individual DMs in a population, known as the followers, respond to uncertainties according to their risk preference types. In the upper level, the leader influences the distribution of the types to induce targeted decisions and steers the follower’s preferences to it. Our analysis centers around the solution concept of approximate Stackelberg equilibrium that yields suboptimal behaviors of the players. We show the existence of the approximate Stackelberg equilibrium. The primitive risk perception gap, defined as the Wasserstein distance between the original and the target type distributions, is important in estimating the optimal design cost. We connect the leader’s optimality compromise on the cost with her ambiguity tolerance on the follower’s approximate solutions leveraging Lipschitzian properties of the lower level solution mapping. To obtain the Stackelberg equilibrium, we reformulate STRIPE into a single-level optimization problem using the spectral representations of law-invariant coherent risk measures. We create a data-driven approach for computation and study its performance guarantees. We apply STRIPE to contract design problems under approximate incentive compatibility. Moreover, we connect STRIPE with meta-learning problems and derive adaptation performance estimates of the meta-parameters.
{"title":"Stackelberg risk preference design","authors":"","doi":"10.1007/s10107-024-02083-2","DOIUrl":"https://doi.org/10.1007/s10107-024-02083-2","url":null,"abstract":"<h3>Abstract</h3> <p>Risk measures are commonly used to capture the risk preferences of decision-makers (DMs). The decisions of DMs can be nudged or manipulated when their risk preferences are influenced by factors such as the availability of information about the uncertainties. This work proposes a Stackelberg risk preference design (STRIPE) problem to capture a designer’s incentive to influence DMs’ risk preferences. STRIPE consists of two levels. In the lower level, individual DMs in a population, known as the followers, respond to uncertainties according to their risk preference types. In the upper level, the leader influences the distribution of the types to induce targeted decisions and steers the follower’s preferences to it. Our analysis centers around the solution concept of approximate Stackelberg equilibrium that yields suboptimal behaviors of the players. We show the existence of the approximate Stackelberg equilibrium. The primitive risk perception gap, defined as the Wasserstein distance between the original and the target type distributions, is important in estimating the optimal design cost. We connect the leader’s optimality compromise on the cost with her ambiguity tolerance on the follower’s approximate solutions leveraging Lipschitzian properties of the lower level solution mapping. To obtain the Stackelberg equilibrium, we reformulate STRIPE into a single-level optimization problem using the spectral representations of law-invariant coherent risk measures. We create a data-driven approach for computation and study its performance guarantees. We apply STRIPE to contract design problems under approximate incentive compatibility. Moreover, we connect STRIPE with meta-learning problems and derive adaptation performance estimates of the meta-parameters. </p>","PeriodicalId":18297,"journal":{"name":"Mathematical Programming","volume":"48 1","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140596179","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mental illnesses are the leading cause of disease burden among children and young people (CYP) globally. Low- and middle-income countries (LMIC) are disproportionately affected. Enhancing mental health literacy (MHL) is one way to combat low levels of help-seeking and effective treatment receipt. We aimed to synthesis evidence about knowledge, beliefs and attitudes of CYP in LMICs about mental illnesses, their treatments and outcomes, evaluating factors that can enhance or impede help-seeking to inform context-specific and developmentally appropriate understandings of MHL. Eight bibliographic databases were searched from inception to July 2020: PsycInfo, EMBASE, Medline (OVID), Scopus, ASSIA (ProQuest), SSCI, SCI (Web of Science) CINAHL PLUS, Social Sciences full text (EBSCO). 58 papers (41 quantitative, 13 qualitative, 4 mixed methods) representing 52 separate studies comprising 36,429 participants with a mean age of 15.3 [10.4-17.4], were appraised and synthesized using narrative synthesis methods. Low levels of recognition and knowledge about mental health problems and illnesses, pervasive levels of stigma and low confidence in professional healthcare services, even when considered a valid treatment option were dominant themes. CYP cited the value of traditional healers and social networks for seeking help. Several important areas were under-researched including the link between specific stigma types and active help-seeking and research is needed to understand more fully the interplay between knowledge, beliefs and attitudes across varied cultural settings. Greater exploration of social networks and the value of collaboration with traditional healers is consistent with promising, yet understudied, areas of community-based MHL interventions combining education and social contact.
{"title":"Mental health literacy in children and adolescents in low- and middle-income countries: a mixed studies systematic review and narrative synthesis.","authors":"Laoise Renwick, Rebecca Pedley, Isobel Johnson, Vicky Bell, Karina Lovell, Penny Bee, Helen Brooks","doi":"10.1007/s00787-022-01997-6","DOIUrl":"10.1007/s00787-022-01997-6","url":null,"abstract":"<p><p>Mental illnesses are the leading cause of disease burden among children and young people (CYP) globally. Low- and middle-income countries (LMIC) are disproportionately affected. Enhancing mental health literacy (MHL) is one way to combat low levels of help-seeking and effective treatment receipt. We aimed to synthesis evidence about knowledge, beliefs and attitudes of CYP in LMICs about mental illnesses, their treatments and outcomes, evaluating factors that can enhance or impede help-seeking to inform context-specific and developmentally appropriate understandings of MHL. Eight bibliographic databases were searched from inception to July 2020: PsycInfo, EMBASE, Medline (OVID), Scopus, ASSIA (ProQuest), SSCI, SCI (Web of Science) CINAHL PLUS, Social Sciences full text (EBSCO). 58 papers (41 quantitative, 13 qualitative, 4 mixed methods) representing 52 separate studies comprising 36,429 participants with a mean age of 15.3 [10.4-17.4], were appraised and synthesized using narrative synthesis methods. Low levels of recognition and knowledge about mental health problems and illnesses, pervasive levels of stigma and low confidence in professional healthcare services, even when considered a valid treatment option were dominant themes. CYP cited the value of traditional healers and social networks for seeking help. Several important areas were under-researched including the link between specific stigma types and active help-seeking and research is needed to understand more fully the interplay between knowledge, beliefs and attitudes across varied cultural settings. Greater exploration of social networks and the value of collaboration with traditional healers is consistent with promising, yet understudied, areas of community-based MHL interventions combining education and social contact.</p>","PeriodicalId":18297,"journal":{"name":"Mathematical Programming","volume":"13 1","pages":"961-985"},"PeriodicalIF":2.2,"publicationDate":"2024-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11032284/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"73281082","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-25DOI: 10.1007/s10107-024-02071-6
Abstract
This work derives upper bounds on the convergence rate of the moment-sum-of-squares hierarchy with correlative sparsity for global minimization of polynomials on compact basic semialgebraic sets. The main conclusion is that both sparse hierarchies based on the Schmüdgen and Putinar Positivstellensätze enjoy a polynomial rate of convergence that depends on the size of the largest clique in the sparsity graph but not on the ambient dimension. Interestingly, the sparse bounds outperform the best currently available bounds for the dense hierarchy when the maximum clique size is sufficiently small compared to the ambient dimension and the performance is measured by the running time of an interior point method required to obtain a bound on the global minimum of a given accuracy.
{"title":"Convergence rates for sums-of-squares hierarchies with correlative sparsity","authors":"","doi":"10.1007/s10107-024-02071-6","DOIUrl":"https://doi.org/10.1007/s10107-024-02071-6","url":null,"abstract":"<h3>Abstract</h3> <p>This work derives upper bounds on the convergence rate of the moment-sum-of-squares hierarchy with correlative sparsity for global minimization of polynomials on compact basic semialgebraic sets. The main conclusion is that both sparse hierarchies based on the Schmüdgen and Putinar Positivstellensätze enjoy a polynomial rate of convergence that depends on the size of the largest clique in the sparsity graph but not on the ambient dimension. Interestingly, the sparse bounds outperform the best currently available bounds for the dense hierarchy when the maximum clique size is sufficiently small compared to the ambient dimension and the performance is measured by the running time of an interior point method required to obtain a bound on the global minimum of a given accuracy.</p>","PeriodicalId":18297,"journal":{"name":"Mathematical Programming","volume":"90 1","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-03-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140298272","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-22DOI: 10.1007/s10107-024-02073-4
Alex L. Wang, Fatma Kılınç-Karzan
This paper introduces a new storage-optimal first-order method, CertSDP, for solving a special class of semidefinite programs (SDPs) to high accuracy. The class of SDPs that we consider, the exact QMP-like SDPs, is characterized by low-rank solutions, a priori knowledge of the restriction of the SDP solution to a small subspace, and standard regularity assumptions such as strict complementarity. Crucially, we show how to use a certificate of strict complementarity to construct a low-dimensional strongly convex minimax problem whose optimizer coincides with a factorization of the SDP optimizer. From an algorithmic standpoint, we show how to construct the necessary certificate and how to solve the minimax problem efficiently. Our algorithms for strongly convex minimax problems with inexact prox maps may be of independent interest. We accompany our theoretical results with preliminary numerical experiments suggesting that CertSDP significantly outperforms current state-of-the-art methods on large sparse exact QMP-like SDPs.
{"title":"Accelerated first-order methods for a class of semidefinite programs","authors":"Alex L. Wang, Fatma Kılınç-Karzan","doi":"10.1007/s10107-024-02073-4","DOIUrl":"https://doi.org/10.1007/s10107-024-02073-4","url":null,"abstract":"<p>This paper introduces a new storage-optimal first-order method, CertSDP, for solving a special class of semidefinite programs (SDPs) to high accuracy. The class of SDPs that we consider, the <i>exact QMP-like SDPs</i>, is characterized by low-rank solutions, <i>a priori</i> knowledge of the restriction of the SDP solution to a small subspace, and standard regularity assumptions such as strict complementarity. Crucially, we show how to use a <i>certificate of strict complementarity</i> to construct a low-dimensional strongly convex minimax problem whose optimizer coincides with a factorization of the SDP optimizer. From an algorithmic standpoint, we show how to construct the necessary certificate and how to solve the minimax problem efficiently. Our algorithms for strongly convex minimax problems with inexact prox maps may be of independent interest. We accompany our theoretical results with preliminary numerical experiments suggesting that CertSDP significantly outperforms current state-of-the-art methods on large sparse exact QMP-like SDPs.\u0000</p>","PeriodicalId":18297,"journal":{"name":"Mathematical Programming","volume":"31 1","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140203212","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-15DOI: 10.1007/s10107-024-02072-5
Abstract
We consider min–max optimization problems for polynomial functions, where a multivariate polynomial is maximized with respect to a subset of variables, and the resulting maximal value is minimized with respect to the remaining variables. When the variables belong to simple sets (e.g., a hypercube, the Euclidean hypersphere, or a ball), we derive a sum-of-squares formulation based on a primal-dual approach. In the simplest setting, we provide a convergence proof when the degree of the relaxation tends to infinity and observe empirically that it can be finitely convergent in several situations. Moreover, our formulation leads to an interesting link with feasibility certificates for polynomial inequalities based on Putinar’s Positivstellensatz.
{"title":"Sum-of-squares relaxations for polynomial min–max problems over simple sets","authors":"","doi":"10.1007/s10107-024-02072-5","DOIUrl":"https://doi.org/10.1007/s10107-024-02072-5","url":null,"abstract":"<h3>Abstract</h3> <p>We consider min–max optimization problems for polynomial functions, where a multivariate polynomial is maximized with respect to a subset of variables, and the resulting maximal value is minimized with respect to the remaining variables. When the variables belong to simple sets (e.g., a hypercube, the Euclidean hypersphere, or a ball), we derive a sum-of-squares formulation based on a primal-dual approach. In the simplest setting, we provide a convergence proof when the degree of the relaxation tends to infinity and observe empirically that it can be finitely convergent in several situations. Moreover, our formulation leads to an interesting link with feasibility certificates for polynomial inequalities based on Putinar’s Positivstellensatz. </p>","PeriodicalId":18297,"journal":{"name":"Mathematical Programming","volume":"3 1","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-03-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140147007","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-14DOI: 10.1007/s10107-024-02076-1
Samuel Burer
Globally optimizing a nonconvex quadratic over the intersection of m balls in (mathbb {R}^n) is known to be polynomial-time solvable for fixed m. Moreover, when (m=1), the standard semidefinite relaxation is exact. When (m=2), it has been shown recently that an exact relaxation can be constructed using a disjunctive semidefinite formulation based essentially on two copies of the (m=1) case. However, there is no known explicit, tractable, exact convex representation for (m ge 3). In this paper, we construct a new, polynomially sized semidefinite relaxation for all m, which does not employ a disjunctive approach. We show that our relaxation is exact for (m=2). Then, for (m ge 3), we demonstrate empirically that it is fast and strong compared to existing relaxations. The key idea of the relaxation is a simple lifting of the original problem into dimension (n, +, 1). Extending this construction: (i) we show that nonconvex quadratic programming over (Vert xVert le min { 1, g + h^T x }) has an exact semidefinite representation; and (ii) we construct a new relaxation for quadratic programming over the intersection of two ellipsoids, which globally solves all instances of a benchmark collection from the literature.
众所周知,对于固定的 m,在 (mathbb {R}^n)中的 m 个球的交点上进行非凸二次函数的全局优化是多项式时间可解的。此外,当 (m=1) 时,标准的半有限松弛是精确的。当(m=2)时,最近有研究表明,可以使用一种基于(m=1)情况的两份分条件半定式来构造精确松弛。然而,对于 (m ge 3) 还没有已知的明确的、可操作的、精确的凸表示。在本文中,我们为所有 m 构建了一个新的、多项式大小的半有限松弛,它没有采用析取方法。我们证明我们的松弛对于(m=2)是精确的。然后,对于 (m ge 3), 我们通过经验证明,与现有的松弛方法相比,我们的松弛方法既快又强。松弛的关键思想是将原始问题简单地提升到维度(n, +, 1)。扩展这个构造:(i)我们证明了在(Vert xVert le min { 1, g + h^T x })上的非凸二次规划有一个精确的半有限表示;(ii)我们为在两个椭圆的交点上的二次规划构造了一个新的松弛,它在全局上解决了文献中一个基准集合的所有实例。
{"title":"A slightly lifted convex relaxation for nonconvex quadratic programming with ball constraints","authors":"Samuel Burer","doi":"10.1007/s10107-024-02076-1","DOIUrl":"https://doi.org/10.1007/s10107-024-02076-1","url":null,"abstract":"<p>Globally optimizing a nonconvex quadratic over the intersection of <i>m</i> balls in <span>(mathbb {R}^n)</span> is known to be polynomial-time solvable for fixed <i>m</i>. Moreover, when <span>(m=1)</span>, the standard semidefinite relaxation is exact. When <span>(m=2)</span>, it has been shown recently that an exact relaxation can be constructed using a disjunctive semidefinite formulation based essentially on two copies of the <span>(m=1)</span> case. However, there is no known explicit, tractable, exact convex representation for <span>(m ge 3)</span>. In this paper, we construct a new, polynomially sized semidefinite relaxation for all <i>m</i>, which does not employ a disjunctive approach. We show that our relaxation is exact for <span>(m=2)</span>. Then, for <span>(m ge 3)</span>, we demonstrate empirically that it is fast and strong compared to existing relaxations. The key idea of the relaxation is a simple lifting of the original problem into dimension <span>(n, +, 1)</span>. Extending this construction: (i) we show that nonconvex quadratic programming over <span>(Vert xVert le min { 1, g + h^T x })</span> has an exact semidefinite representation; and (ii) we construct a new relaxation for quadratic programming over the intersection of two ellipsoids, which globally solves all instances of a benchmark collection from the literature.</p>","PeriodicalId":18297,"journal":{"name":"Mathematical Programming","volume":"35 1","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140146947","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-13DOI: 10.1007/s10107-024-02075-2
Tianyi Lin, Michael I. Jordan
This paper settles an open and challenging question pertaining to the design of simple and optimal high-order methods for solving smooth and monotone variational inequalities (VIs). A VI involves finding (x^star in {mathcal {X}}) such that (langle F(x), x - x^star rangle ge 0) for all (x in {mathcal {X}}). We consider the setting in which (F: {mathbb {R}}^d rightarrow {mathbb {R}}^d) is smooth with up to ((p-1)^{text {th}})-order derivatives. For (p = 2), the cubic regularization of Newton’s method has been extended to VIs with a global rate of (O(epsilon ^{-1})) (Nesterov in Cubic regularization of Newton’s method for convex problems with constraints, Tech. rep., Université catholique de Louvain, Center for Operations Research and Econometrics (CORE), 2006). An improved rate of (O(epsilon ^{-2/3}log log (1/epsilon ))) can be obtained via an alternative second-order method, but this method requires a nontrivial line-search procedure as an inner loop. Similarly, the existing high-order methods based on line-search procedures have been shown to achieve a rate of (O(epsilon ^{-2/(p+1)}log log (1/epsilon ))) (Bullins and Lai in SIAM J Optim 32(3):2208–2229, 2022; Jiang and Mokhtari in Generalized optimistic methods for convex–concave saddle point problems, 2022; Lin and Jordan in Math Oper Res 48(4):2353–2382, 2023). As emphasized by Nesterov (Lectures on convex optimization, vol 137, Springer, Berlin, 2018), however, such procedures do not necessarily imply the practical applicability in large-scale applications, and it is desirable to complement these results with a simple high-order VI method that retains the optimality of the more complex methods. We propose a (p^{text {th}})-order method that does not require any line search procedure and provably converges to a weak solution at a rate of (O(epsilon ^{-2/(p+1)})). We prove that our (p^{text {th}})-order method is optimal in the monotone setting by establishing a lower bound of (Omega (epsilon ^{-2/(p+1)})) under a generalized linear span assumption. A restarted version of our (p^{text {th}})-order method attains a linear rate for smooth and (p^{text {th}})-order uniformly monotone VIs and another restarted version of our (p^{text {th}})-order method attains a local superlinear rate for smooth and strongly monotone VIs. Further, the similar (p^{text {th}})-order method achieves a global rate of (O(epsilon ^{-2/p})) for solving smooth and nonmonotone VIs satisfying the Minty condition. Two restarted versions attain a global linear rate under additional (p^{text {th}})-order uniform Minty condition and a local superlinear rate under additional strong Minty condition.
本文解决了一个开放且具有挑战性的问题,即设计简单且最优的高阶方法来求解平滑且单调的变分不等式(VIs)。变分不等式涉及找到 (x^starin {mathcal {X}}) such that (angle F(x), x - x^starrangle ge 0) for all (x in {mathcal {X}}).我们考虑这样一种情况:F: {mathbb {R}}^d rightarrow {mathbb {R}}^d) 是光滑的,最多有((p-1)^{text {th}})阶导数。对于 (p = 2), 牛顿方法的立方正则化已经扩展到 VIs,其全局速率为 (O(epsilon ^{-1}))(Nesterov 在 Cubic regularization of Newton's method for convex problems with constraints, Tech. rep.通过另一种二阶方法可以得到一个改进的速率(O(epsilon ^{-2/3}log log (1/epsilon ))) ,但这种方法需要一个非线性的线性搜索过程作为内循环。同样,现有的基于线性搜索过程的高阶方法已经被证明可以达到 (O(epsilon ^{-2/(p+1)}log log (1/epsilon ))) (Bullins 和 Lai 在 SIAM J Optim 32(3):2208-2229, 2022; Jiang and Mokhtari in Generalized optimistic methods for convex-concave saddle point problems, 2022; Lin and Jordan in Math Oper Res 48(4):2353-2382, 2023)。然而,正如涅斯捷罗夫(Lectures on convex optimization, vol 137, Springer, Berlin, 2018)所强调的,这些程序并不一定意味着在大规模应用中的实际适用性,我们希望用一种简单的高阶 VI 方法来补充这些结果,同时保留更复杂方法的最优性。我们提出了一种 (p^{text {th}})阶方法,它不需要任何线性搜索过程,并能以 (O(epsilon ^{-2/(p+1)})) 的速率收敛到弱解。我们通过在广义线性跨度假设下建立一个 (Omega (epsilon ^{-2/(p+1)})) 的下限,证明我们的 (p^{text {th}})-order 方法在单调设置中是最优的。我们的(p^{text {th}})阶方法的重启版本对于平滑和(p^{text {th}})阶均匀单调VI达到了线性速率,我们的(p^{text {th}})阶方法的另一个重启版本对于平滑和强单调VI达到了局部超线性速率。此外,类似的(p^{text {th}})阶方法在求解满足Minty条件的平滑和非单调VI时达到了(O(epsilon ^{-2/p}))的全局速率。两个重启版本在附加的 (p^{text {th}})-order uniform Minty 条件下达到了全局线性速率,在附加的强 Minty 条件下达到了局部超线性速率。
{"title":"Perseus: a simple and optimal high-order method for variational inequalities","authors":"Tianyi Lin, Michael I. Jordan","doi":"10.1007/s10107-024-02075-2","DOIUrl":"https://doi.org/10.1007/s10107-024-02075-2","url":null,"abstract":"<p>This paper settles an open and challenging question pertaining to the design of simple and optimal high-order methods for solving smooth and monotone variational inequalities (VIs). A VI involves finding <span>(x^star in {mathcal {X}})</span> such that <span>(langle F(x), x - x^star rangle ge 0)</span> for all <span>(x in {mathcal {X}})</span>. We consider the setting in which <span>(F: {mathbb {R}}^d rightarrow {mathbb {R}}^d)</span> is smooth with up to <span>((p-1)^{text {th}})</span>-order derivatives. For <span>(p = 2)</span>, the cubic regularization of Newton’s method has been extended to VIs with a global rate of <span>(O(epsilon ^{-1}))</span> (Nesterov in Cubic regularization of Newton’s method for convex problems with constraints, Tech. rep., Université catholique de Louvain, Center for Operations Research and Econometrics (CORE), 2006). An improved rate of <span>(O(epsilon ^{-2/3}log log (1/epsilon )))</span> can be obtained via an alternative second-order method, but this method requires a nontrivial line-search procedure as an inner loop. Similarly, the existing high-order methods based on line-search procedures have been shown to achieve a rate of <span>(O(epsilon ^{-2/(p+1)}log log (1/epsilon )))</span> (Bullins and Lai in SIAM J Optim 32(3):2208–2229, 2022; Jiang and Mokhtari in Generalized optimistic methods for convex–concave saddle point problems, 2022; Lin and Jordan in Math Oper Res 48(4):2353–2382, 2023). As emphasized by Nesterov (Lectures on convex optimization, vol 137, Springer, Berlin, 2018), however, such procedures do not necessarily imply the practical applicability in large-scale applications, and it is desirable to complement these results with a simple high-order VI method that retains the optimality of the more complex methods. We propose a <span>(p^{text {th}})</span>-order method that does <i>not</i> require any line search procedure and provably converges to a weak solution at a rate of <span>(O(epsilon ^{-2/(p+1)}))</span>. We prove that our <span>(p^{text {th}})</span>-order method is optimal in the monotone setting by establishing a lower bound of <span>(Omega (epsilon ^{-2/(p+1)}))</span> under a generalized linear span assumption. A restarted version of our <span>(p^{text {th}})</span>-order method attains a linear rate for smooth and <span>(p^{text {th}})</span>-order uniformly monotone VIs and another restarted version of our <span>(p^{text {th}})</span>-order method attains a local superlinear rate for smooth and strongly monotone VIs. Further, the similar <span>(p^{text {th}})</span>-order method achieves a global rate of <span>(O(epsilon ^{-2/p}))</span> for solving smooth and nonmonotone VIs satisfying the Minty condition. Two restarted versions attain a global linear rate under additional <span>(p^{text {th}})</span>-order uniform Minty condition and a local superlinear rate under additional strong Minty condition.</p>","PeriodicalId":18297,"journal":{"name":"Mathematical Programming","volume":"16 1","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140128068","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}