A stochastic algorithm is proposed, analyzed, and tested experimentally for solving continuous optimization problems with nonlinear equality constraints. It is assumed that constraint function and derivative values can be computed but that only stochastic approximations are available for the objective function and its derivatives. The algorithm is of the sequential quadratic optimization variety. Distinguishing features of the algorithm are that it only employs stochastic objective gradient estimates that satisfy a relatively weak set of assumptions (while using neither objective function values nor estimates of them) and that it allows inexact subproblem solutions to be employed, the latter of which is particularly useful in large-scale settings when the matrices defining the subproblems are too large to form and/or factorize. Conditions are imposed on the inexact subproblem solutions that account for the fact that only stochastic objective gradient estimates are employed. Convergence results are established for the method. Numerical experiments show that the proposed method vastly outperforms a stochastic subgradient method and can outperform an alternative sequential quadratic programming algorithm that employs highly accurate subproblem solutions in every iteration. Funding: This material is based upon work supported by the National Science Foundation [Awards CCF-1740796 and CCF-2139735] and the Office of Naval Research [Award N00014-21-1-2532].
{"title":"A Stochastic Inexact Sequential Quadratic Optimization Algorithm for Nonlinear Equality-Constrained Optimization","authors":"Frank E. Curtis, Daniel P. Robinson, Baoyu Zhou","doi":"10.1287/ijoo.2022.0008","DOIUrl":"https://doi.org/10.1287/ijoo.2022.0008","url":null,"abstract":"A stochastic algorithm is proposed, analyzed, and tested experimentally for solving continuous optimization problems with nonlinear equality constraints. It is assumed that constraint function and derivative values can be computed but that only stochastic approximations are available for the objective function and its derivatives. The algorithm is of the sequential quadratic optimization variety. Distinguishing features of the algorithm are that it only employs stochastic objective gradient estimates that satisfy a relatively weak set of assumptions (while using neither objective function values nor estimates of them) and that it allows inexact subproblem solutions to be employed, the latter of which is particularly useful in large-scale settings when the matrices defining the subproblems are too large to form and/or factorize. Conditions are imposed on the inexact subproblem solutions that account for the fact that only stochastic objective gradient estimates are employed. Convergence results are established for the method. Numerical experiments show that the proposed method vastly outperforms a stochastic subgradient method and can outperform an alternative sequential quadratic programming algorithm that employs highly accurate subproblem solutions in every iteration. Funding: This material is based upon work supported by the National Science Foundation [Awards CCF-1740796 and CCF-2139735] and the Office of Naval Research [Award N00014-21-1-2532].","PeriodicalId":73382,"journal":{"name":"INFORMS journal on optimization","volume":"5 6","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-05-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141100955","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper addresses problems of two-stage optimization under binary uncertainty. We define a scenario-based robust optimization (ScRO) formulation that combines principles of stochastic optimization (by constructing probabilistic scenarios) and robust optimization (by protecting against adversarial perturbations within discrete uncertainty sets). To solve it, we develop a sparse row generation algorithm that iterates between a master problem (which provides a lower bound based on minimal uncertainty sets) and a history-based subproblem (which generates an upper bound and updates minimal uncertainty sets). We generate scenarios and uncertainty sets from element-wise probabilities using a deviation likelihood method or from historical samples using a sample clustering approach. Using public data sets, results suggest that (i) our ScRO formulation outperforms benchmarks based on deterministic, stochastic, and robust optimization; (ii) our deviation likelihood and sample clustering approaches outperform scenario generation baselines; and (iii) our sparse row generation algorithm outperforms off-the-shelf implementation and state-of-the-art cutting plane benchmarks. An application to a real-world ambulance dispatch case study suggests that the proposed modeling and algorithmic approach can reduce the number of late responses by more than 25%. Funding: K. Wang’s research was supported by the National Natural Science Foundation of China [Grants 72322002, 52221005 and 52220105001]
{"title":"Scenario-Based Robust Optimization for Two-Stage Decision Making Under Binary Uncertainty","authors":"Kai Wang, Mehmet Aydemir, Alexandre Jacquillat","doi":"10.1287/ijoo.2020.0038","DOIUrl":"https://doi.org/10.1287/ijoo.2020.0038","url":null,"abstract":"This paper addresses problems of two-stage optimization under binary uncertainty. We define a scenario-based robust optimization (ScRO) formulation that combines principles of stochastic optimization (by constructing probabilistic scenarios) and robust optimization (by protecting against adversarial perturbations within discrete uncertainty sets). To solve it, we develop a sparse row generation algorithm that iterates between a master problem (which provides a lower bound based on minimal uncertainty sets) and a history-based subproblem (which generates an upper bound and updates minimal uncertainty sets). We generate scenarios and uncertainty sets from element-wise probabilities using a deviation likelihood method or from historical samples using a sample clustering approach. Using public data sets, results suggest that (i) our ScRO formulation outperforms benchmarks based on deterministic, stochastic, and robust optimization; (ii) our deviation likelihood and sample clustering approaches outperform scenario generation baselines; and (iii) our sparse row generation algorithm outperforms off-the-shelf implementation and state-of-the-art cutting plane benchmarks. An application to a real-world ambulance dispatch case study suggests that the proposed modeling and algorithmic approach can reduce the number of late responses by more than 25%. Funding: K. Wang’s research was supported by the National Natural Science Foundation of China [Grants 72322002, 52221005 and 52220105001]","PeriodicalId":73382,"journal":{"name":"INFORMS journal on optimization","volume":"119 6","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135872794","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We consider a repeated newsvendor problem in which the inventory manager has no prior information about the demand and can access only censored/sales data. In analogy to multiarmed bandit problems, the manager needs to simultaneously “explore” and “exploit” with inventory decisions in order to minimize the cumulative cost. Our goal is to understand the hardness of the problem disentangled from any probabilistic assumptions on the demand sequence—importantly, independence or time stationarity—and, correspondingly, to develop policies that perform well with respect to the regret criterion. We design a cost estimator that is tailored to the special structure of the censoring problem, and we show that, if coupled with the classic exponentially weighted forecaster, it achieves optimal scaling of the expected regret (up to logarithmic factors) with respect to both the number of time periods and available actions. This result also leads to two important insights: the benefit from “information stalking” as well as the cost of censoring are both negligible, at least in terms of the regret. We demonstrate the flexibility of our technique by combining it with the fixed share forecaster to provide strong guarantees in terms of tracking regret, a powerful notion of regret that uses a large class of time-varying action sequences as benchmark. Numerical experiments suggest that the resulting policy outperforms existing policies (that are tailored to or facilitated by time stationarity) on nonstationary demand models with time-varying noise, trend, and seasonality components. Finally, we consider the “combinatorial” version of the repeated newsvendor problem, that is, single-warehouse, multiretailer inventory management of a perishable product. We extend the proposed approach so that, again, it achieves near-optimal performance in terms of the regret. Funding: G. Lugosi was supported by the Spanish Ministry of Economy, Industry and Competitiveness [Grant MTM2015-67304-P (AEI/FEDER, UE)]. M. G. Markakis was supported by the Spanish Ministry of Economy and Competitiveness [Grant ECO2016-75905-R (AEI/FEDER, UE)] and a Juan de la Cierva fellowship as well as the Spanish Ministry of Science and Innovation through a Ramón y Cajal fellowship. G. Neu was supported by the UPFellows Fellowship (Marie Curie COFUND program) [Grant 600387]. Supplemental Material: The e-companion is available at https://doi.org/10.1287/ijoo.2022.0017 .
我们考虑一个重复报贩问题，其中库存管理人员没有关于需求的先验信息，只能访问审查/销售数据。与多武装盗匪问题类似，管理者需要同时“探索”和“利用”库存决策，以最小化累积成本。我们的目标是理解从需求序列的任何概率假设(重要的是独立性或时间平稳性)中解脱出来的问题的难度，并相应地制定相对于后悔标准表现良好的策略。我们设计了一个针对审查问题的特殊结构量身定制的成本估计器，并且我们表明，如果与经典的指数加权预测器相结合，它可以根据时间周期和可用操作的数量实现预期后悔(高达对数因子)的最佳缩放。这个结果还带来了两个重要的启示:“信息跟踪”的好处和审查的成本都可以忽略不计，至少在后悔方面是这样。我们通过将其与固定份额预测器相结合来展示我们技术的灵活性，从而在跟踪后悔方面提供强有力的保证，这是一种使用大量时变动作序列作为基准的强大的后悔概念。数值实验表明，在具有时变噪声、趋势和季节性成分的非平稳需求模型上，所得政策优于现有政策(根据时间平稳性量身定制或促进)。最后，我们考虑重复报贩问题的“组合”版本，即易腐产品的单仓库、多零售商库存管理。我们扩展了所提出的方法，因此，再一次，它在遗憾方面达到了接近最佳的性能。资助:G. Lugosi由西班牙经济、工业和竞争力部资助[Grant MTM2015-67304-P (AEI/FEDER, UE)]。m.g. Markakis得到了西班牙经济与竞争力部[Grant ECO2016-75905-R (AEI/FEDER, UE)]、Juan de la Cierva奖学金以及西班牙科学与创新部Ramón y Cajal奖学金的支持。G. Neu得到了UPFellows Fellowship (Marie Curie COFUND program)的资助[Grant 600387]。补充材料:电子伴侣可在https://doi.org/10.1287/ijoo.2022.0017上获得。
{"title":"On the Hardness of Learning from Censored and Nonstationary Demand","authors":"Gábor Lugosi, Mihalis G. Markakis, Gergely Neu","doi":"10.1287/ijoo.2022.0017","DOIUrl":"https://doi.org/10.1287/ijoo.2022.0017","url":null,"abstract":"We consider a repeated newsvendor problem in which the inventory manager has no prior information about the demand and can access only censored/sales data. In analogy to multiarmed bandit problems, the manager needs to simultaneously “explore” and “exploit” with inventory decisions in order to minimize the cumulative cost. Our goal is to understand the hardness of the problem disentangled from any probabilistic assumptions on the demand sequence—importantly, independence or time stationarity—and, correspondingly, to develop policies that perform well with respect to the regret criterion. We design a cost estimator that is tailored to the special structure of the censoring problem, and we show that, if coupled with the classic exponentially weighted forecaster, it achieves optimal scaling of the expected regret (up to logarithmic factors) with respect to both the number of time periods and available actions. This result also leads to two important insights: the benefit from “information stalking” as well as the cost of censoring are both negligible, at least in terms of the regret. We demonstrate the flexibility of our technique by combining it with the fixed share forecaster to provide strong guarantees in terms of tracking regret, a powerful notion of regret that uses a large class of time-varying action sequences as benchmark. Numerical experiments suggest that the resulting policy outperforms existing policies (that are tailored to or facilitated by time stationarity) on nonstationary demand models with time-varying noise, trend, and seasonality components. Finally, we consider the “combinatorial” version of the repeated newsvendor problem, that is, single-warehouse, multiretailer inventory management of a perishable product. We extend the proposed approach so that, again, it achieves near-optimal performance in terms of the regret. Funding: G. Lugosi was supported by the Spanish Ministry of Economy, Industry and Competitiveness [Grant MTM2015-67304-P (AEI/FEDER, UE)]. M. G. Markakis was supported by the Spanish Ministry of Economy and Competitiveness [Grant ECO2016-75905-R (AEI/FEDER, UE)] and a Juan de la Cierva fellowship as well as the Spanish Ministry of Science and Innovation through a Ramón y Cajal fellowship. G. Neu was supported by the UPFellows Fellowship (Marie Curie COFUND program) [Grant 600387]. Supplemental Material: The e-companion is available at https://doi.org/10.1287/ijoo.2022.0017 .","PeriodicalId":73382,"journal":{"name":"INFORMS journal on optimization","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135386695","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Christopher Muir, Luke Marshall, Alejandro Toriello
Motivated by applications in cloud computing, we study a temporal bin packing problem with jobs that occupy half of a bin’s capacity. An instance is given by a set of jobs, each with a start and end time during which it must be processed (i.e., assigned to a bin). A bin can accommodate two jobs simultaneously, and the objective is an assignment that minimizes the time-averaged number of open or active bins over the horizon; this problem is known to be NP hard. We demonstrate that a well-known “static” lower bound may have a significant gap even in relatively simple instances, which motivates us to introduce a novel combinatorial lower bound and an integer programming formulation, both based on an interpretation of the model as a series of connected matching problems. We theoretically compare the static bound, the new matching-based bounds, and various linear programming bounds. We perform a computational study using both synthetic and application-based instances and show that our bounds offer significant improvement over existing methods, particularly for sparse instances. Funding: This work was supported by the National Science Foundation [Grants CMMI-1552479 and NSF GRFP]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/ijoo.2023.0002 .
{"title":"Temporal Bin Packing with Half-Capacity Jobs","authors":"Christopher Muir, Luke Marshall, Alejandro Toriello","doi":"10.1287/ijoo.2023.0002","DOIUrl":"https://doi.org/10.1287/ijoo.2023.0002","url":null,"abstract":"Motivated by applications in cloud computing, we study a temporal bin packing problem with jobs that occupy half of a bin’s capacity. An instance is given by a set of jobs, each with a start and end time during which it must be processed (i.e., assigned to a bin). A bin can accommodate two jobs simultaneously, and the objective is an assignment that minimizes the time-averaged number of open or active bins over the horizon; this problem is known to be NP hard. We demonstrate that a well-known “static” lower bound may have a significant gap even in relatively simple instances, which motivates us to introduce a novel combinatorial lower bound and an integer programming formulation, both based on an interpretation of the model as a series of connected matching problems. We theoretically compare the static bound, the new matching-based bounds, and various linear programming bounds. We perform a computational study using both synthetic and application-based instances and show that our bounds offer significant improvement over existing methods, particularly for sparse instances. Funding: This work was supported by the National Science Foundation [Grants CMMI-1552479 and NSF GRFP]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/ijoo.2023.0002 .","PeriodicalId":73382,"journal":{"name":"INFORMS journal on optimization","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135817386","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The authors conceived of the bisection protocol during a research meeting discussing recent political redistricting literature, in particular, the I-cut-you-freeze protocol preprint. After establishing the theoretical results for the continuous nongeometric setting, they discussed ways to implement both protocols on real-world data, culminating in the Iowa case study and computational experiments with 17 other states.
{"title":"A Bisection Protocol for Political Redistricting","authors":"Ian G. Ludden, Rahul Swamy, D. King, S. Jacobson","doi":"10.1287/ijoo.2022.0084","DOIUrl":"https://doi.org/10.1287/ijoo.2022.0084","url":null,"abstract":"The authors conceived of the bisection protocol during a research meeting discussing recent political redistricting literature, in particular, the I-cut-you-freeze protocol preprint. After establishing the theoretical results for the continuous nongeometric setting, they discussed ways to implement both protocols on real-world data, culminating in the Iowa case study and computational experiments with 17 other states.","PeriodicalId":73382,"journal":{"name":"INFORMS journal on optimization","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47544022","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Nicolas Busch, Tobias Crönert, Stefan Minner, Moritz Rettinger, Burakhan Sel
As the number of exchange-traded commodity contracts and their volatility increase, risk management through financial hedging gains importance for commodity-purchasing firms. Existing data-driven optimization approaches for hedging decisions include linear regression-based techniques. As such, they assume linear price–feature relationships and, thus, do not automatically detect nonlinear feature effects. We propose an alternative, nonlinear data-driven approach to commodity procurement based on deep learning. The prescriptive algorithm uses artificial neural networks to allow for universal approximation and requires no a priori knowledge regarding underlying price processes. We reformulate the periodic review procurement problem as a multilabel time series classification problem as the optimal bang-bang type procurement policy allows us to treat the hedging decision for each demand period as an individual subproblem that is independent of the other periods. Thereby, we are differentiating between optimal and suboptimal hedging decisions in each period and introduce a novel opportunity cost–sensitive loss function. We train maximum likelihood classifiers based on different deep learning architectures and test their performance in numerical experiments and case studies for natural gas, crude oil, nickel, and copper procurement. We show comparable performance to the state of the art for linear price–feature relationships and considerable advantages in the nonlinear case. Funding: Financial support received through the DFG as part of the AdONE GRK2201 [Grant 277991500] is gratefully acknowledged.
{"title":"Deep Learning for Commodity Procurement: Nonlinear Data-Driven Optimization of Hedging Decisions","authors":"Nicolas Busch, Tobias Crönert, Stefan Minner, Moritz Rettinger, Burakhan Sel","doi":"10.1287/ijoo.2022.0086","DOIUrl":"https://doi.org/10.1287/ijoo.2022.0086","url":null,"abstract":"As the number of exchange-traded commodity contracts and their volatility increase, risk management through financial hedging gains importance for commodity-purchasing firms. Existing data-driven optimization approaches for hedging decisions include linear regression-based techniques. As such, they assume linear price–feature relationships and, thus, do not automatically detect nonlinear feature effects. We propose an alternative, nonlinear data-driven approach to commodity procurement based on deep learning. The prescriptive algorithm uses artificial neural networks to allow for universal approximation and requires no a priori knowledge regarding underlying price processes. We reformulate the periodic review procurement problem as a multilabel time series classification problem as the optimal bang-bang type procurement policy allows us to treat the hedging decision for each demand period as an individual subproblem that is independent of the other periods. Thereby, we are differentiating between optimal and suboptimal hedging decisions in each period and introduce a novel opportunity cost–sensitive loss function. We train maximum likelihood classifiers based on different deep learning architectures and test their performance in numerical experiments and case studies for natural gas, crude oil, nickel, and copper procurement. We show comparable performance to the state of the art for linear price–feature relationships and considerable advantages in the nonlinear case. Funding: Financial support received through the DFG as part of the AdONE GRK2201 [Grant 277991500] is gratefully acknowledged.","PeriodicalId":73382,"journal":{"name":"INFORMS journal on optimization","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136309435","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Peyman Kafaei, Quentin Cappart, Nicolas Chapados, H. Pouya, Louis-Martin Rousseau
With the rapid developments in communication systems, and considering their dynamic nature, all-optical networks are becoming increasingly complex. This study proposes a novel method based on deep reinforcement learning for the routing and wavelength assignment problem in all-optical wavelength-decision-multiplexing networks. We consider dynamic incoming requests, in which their arrival and holding times are not known in advance. The objective is to devise a strategy that minimizes the number of rejected packages due to the lack of resources in the long term. We use graph neural networks to capture crucial latent information from the graph-structured input to develop the optimal strategy. The proposed deep reinforcement learning algorithm selects a route and a wavelength simultaneously for each incoming traffic connection as they arrive. The results demonstrate that the learned agent outperforms the methods used in practice and can be generalized on network topologies that did not participate in training.
{"title":"Dynamic Routing and Wavelength Assignment with Reinforcement Learning","authors":"Peyman Kafaei, Quentin Cappart, Nicolas Chapados, H. Pouya, Louis-Martin Rousseau","doi":"10.1287/ijoo.2023.0092","DOIUrl":"https://doi.org/10.1287/ijoo.2023.0092","url":null,"abstract":"With the rapid developments in communication systems, and considering their dynamic nature, all-optical networks are becoming increasingly complex. This study proposes a novel method based on deep reinforcement learning for the routing and wavelength assignment problem in all-optical wavelength-decision-multiplexing networks. We consider dynamic incoming requests, in which their arrival and holding times are not known in advance. The objective is to devise a strategy that minimizes the number of rejected packages due to the lack of resources in the long term. We use graph neural networks to capture crucial latent information from the graph-structured input to develop the optimal strategy. The proposed deep reinforcement learning algorithm selects a route and a wavelength simultaneously for each incoming traffic connection as they arrive. The results demonstrate that the learned agent outperforms the methods used in practice and can be generalized on network topologies that did not participate in training.","PeriodicalId":73382,"journal":{"name":"INFORMS journal on optimization","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-06-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42660672","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The matrix sensing problem is an important low-rank optimization problem that has found a wide range of applications, such as matrix completion, phase synchornization/retrieval, robust principal component analysis (PCA), and power system state estimation. In this work, we focus on the general matrix sensing problem with linear measurements that are corrupted by random noise. We investigate the scenario where the search rank r is equal to the true rank [Formula: see text] of the unknown ground truth (the exact parametrized case), as well as the scenario where r is greater than [Formula: see text] (the overparametrized case). We quantify the role of the restricted isometry property (RIP) in shaping the landscape of the nonconvex factorized formulation and assisting with the success of local search algorithms. First, we develop a global guarantee on the maximum distance between an arbitrary local minimizer of the nonconvex problem and the ground truth under the assumption that the RIP constant is smaller than [Formula: see text]. We then present a local guarantee for problems with an arbitrary RIP constant, which states that any local minimizer is either considerably close to the ground truth or far away from it. More importantly, we prove that this noisy, overparametrized problem exhibits the strict saddle property, which leads to the global convergence of perturbed gradient descent algorithm in polynomial time. The results of this work provide a comprehensive understanding of the geometric landscape of the matrix sensing problem in the noisy and overparametrized regime. Funding: This work was supported by grants from the National Science Foundation, Office of Naval Research, Air Force Office of Scientific Research, and Army Research Office.
{"title":"Geometric Analysis of Noisy Low-Rank Matrix Recovery in the Exact Parametrized and the Overparametrized Regimes","authors":"Ziye Ma, Yingjie Bi, J. Lavaei, S. Sojoudi","doi":"10.1287/ijoo.2023.0090","DOIUrl":"https://doi.org/10.1287/ijoo.2023.0090","url":null,"abstract":"The matrix sensing problem is an important low-rank optimization problem that has found a wide range of applications, such as matrix completion, phase synchornization/retrieval, robust principal component analysis (PCA), and power system state estimation. In this work, we focus on the general matrix sensing problem with linear measurements that are corrupted by random noise. We investigate the scenario where the search rank r is equal to the true rank [Formula: see text] of the unknown ground truth (the exact parametrized case), as well as the scenario where r is greater than [Formula: see text] (the overparametrized case). We quantify the role of the restricted isometry property (RIP) in shaping the landscape of the nonconvex factorized formulation and assisting with the success of local search algorithms. First, we develop a global guarantee on the maximum distance between an arbitrary local minimizer of the nonconvex problem and the ground truth under the assumption that the RIP constant is smaller than [Formula: see text]. We then present a local guarantee for problems with an arbitrary RIP constant, which states that any local minimizer is either considerably close to the ground truth or far away from it. More importantly, we prove that this noisy, overparametrized problem exhibits the strict saddle property, which leads to the global convergence of perturbed gradient descent algorithm in polynomial time. The results of this work provide a comprehensive understanding of the geometric landscape of the matrix sensing problem in the noisy and overparametrized regime. Funding: This work was supported by grants from the National Science Foundation, Office of Naval Research, Air Force Office of Scientific Research, and Army Research Office.","PeriodicalId":73382,"journal":{"name":"INFORMS journal on optimization","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-04-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46590619","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kevin-Martin Aigner, Andreas Bärmann, Kristin Braun, F. Liers, S. Pokutta, Oskar Schneider, Kartikey Sharma, Sebastian Tschuppik
Stochastic optimization (SO) is a classical approach for optimization under uncertainty that typically requires knowledge about the probability distribution of uncertain parameters. Because the latter is often unknown, distributionally robust optimization (DRO) provides a strong alternative that determines the best guaranteed solution over a set of distributions (ambiguity set). In this work, we present an approach for DRO over time that uses online learning and scenario observations arriving as a data stream to learn more about the uncertainty. Our robust solutions adapt over time and reduce the cost of protection with shrinking ambiguity. For various kinds of ambiguity sets, the robust solutions converge to the SO solution. Our algorithm achieves the optimization and learning goals without solving the DRO problem exactly at any step. We also provide a regret bound for the quality of the online strategy that converges at a rate of [Formula: see text], where T is the number of iterations. Furthermore, we illustrate the effectiveness of our procedure by numerical experiments on mixed-integer optimization instances from popular benchmark libraries and give practical examples stemming from telecommunications and routing. Our algorithm is able to solve the DRO over time problem significantly faster than standard reformulations. Funding: This work was supported by Deutsche Forschungsgemeinschaft (DFG): Projects B06 and B10 in CRC TRR 154 and Project-ID 416229255 - SFB 1411 and Federal Ministry for Economic Affairs and Energy, Germany [Grant 03EI1036A]. Supplemental Material: The e-companion is available at https://doi.org/10.1287/ijoo.2023.0091 .
{"title":"Data-Driven Distributionally Robust Optimization over Time","authors":"Kevin-Martin Aigner, Andreas Bärmann, Kristin Braun, F. Liers, S. Pokutta, Oskar Schneider, Kartikey Sharma, Sebastian Tschuppik","doi":"10.1287/ijoo.2023.0091","DOIUrl":"https://doi.org/10.1287/ijoo.2023.0091","url":null,"abstract":"Stochastic optimization (SO) is a classical approach for optimization under uncertainty that typically requires knowledge about the probability distribution of uncertain parameters. Because the latter is often unknown, distributionally robust optimization (DRO) provides a strong alternative that determines the best guaranteed solution over a set of distributions (ambiguity set). In this work, we present an approach for DRO over time that uses online learning and scenario observations arriving as a data stream to learn more about the uncertainty. Our robust solutions adapt over time and reduce the cost of protection with shrinking ambiguity. For various kinds of ambiguity sets, the robust solutions converge to the SO solution. Our algorithm achieves the optimization and learning goals without solving the DRO problem exactly at any step. We also provide a regret bound for the quality of the online strategy that converges at a rate of [Formula: see text], where T is the number of iterations. Furthermore, we illustrate the effectiveness of our procedure by numerical experiments on mixed-integer optimization instances from popular benchmark libraries and give practical examples stemming from telecommunications and routing. Our algorithm is able to solve the DRO over time problem significantly faster than standard reformulations. Funding: This work was supported by Deutsche Forschungsgemeinschaft (DFG): Projects B06 and B10 in CRC TRR 154 and Project-ID 416229255 - SFB 1411 and Federal Ministry for Economic Affairs and Energy, Germany [Grant 03EI1036A]. Supplemental Material: The e-companion is available at https://doi.org/10.1287/ijoo.2023.0091 .","PeriodicalId":73382,"journal":{"name":"INFORMS journal on optimization","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47231198","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：