Pub Date : 2024-09-18DOI: 10.1007/s10898-024-01427-8
Chieu Thanh Nguyen, Jan Harold Alcantara, Zijun Hao, Jein-Shan Chen
In this paper, we propose a smoothing penalty approach for solving the second-order cone complementarity problem (SOCCP). The SOCCP is approximated by a smooth nonlinear equation with penalization parameter. We show that any solution sequence of the approximating equations converges to the solution of the SOCCP under the assumption that the associated function of the SOCCP satisfies a uniform Cartesian-type property. We present a corresponding algorithm for solving the SOCCP based on this smoothing penalty approach, and we demonstrate the efficiency of our method for solving linear, nonlinear and tensor complementarity problems in the second-order cone setting.
{"title":"Smoothing penalty approach for solving second-order cone complementarity problems","authors":"Chieu Thanh Nguyen, Jan Harold Alcantara, Zijun Hao, Jein-Shan Chen","doi":"10.1007/s10898-024-01427-8","DOIUrl":"https://doi.org/10.1007/s10898-024-01427-8","url":null,"abstract":"<p>In this paper, we propose a smoothing penalty approach for solving the second-order cone complementarity problem (SOCCP). The SOCCP is approximated by a smooth nonlinear equation with penalization parameter. We show that any solution sequence of the approximating equations converges to the solution of the SOCCP under the assumption that the associated function of the SOCCP satisfies a uniform Cartesian-type property. We present a corresponding algorithm for solving the SOCCP based on this smoothing penalty approach, and we demonstrate the efficiency of our method for solving linear, nonlinear and tensor complementarity problems in the second-order cone setting.</p>","PeriodicalId":15961,"journal":{"name":"Journal of Global Optimization","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2024-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142248767","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-17DOI: 10.1007/s10898-024-01393-1
Fernando Dias, David Rey
To guarantee the safety of flight operations, decision-support systems for air traffic control must be able to improve the usage of airspace capacity and handle increasing demand. This study addresses the aircraft conflict avoidance and trajectory recovery problem. The problem of finding the least deviation conflict-free aircraft trajectories that guarantee the return to a target waypoint is highly complex due to the nature of the nonlinear trajectories that are sought. We present a two-stage iterative algorithm that first solves initial conflicts by manipulating their speed and heading control and then identifying each aircraft’s optimal time to recover its trajectory towards their nominal one. We extend existing mixed-integer programming formulations by modelling speed and heading control as continuous variables while recovery time is treated as a discrete variable. We develop a novel iterative approach which shows that the trajectory recovery costs can be anticipated by inducing avoidance trajectories with higher deviation, therefore obtaining earlier recovery time within a few iterations. Numerical results on benchmark conflict resolution problems show that this approach can solve instances with up to 30 aircraft within 10 min.
{"title":"Aircraft conflict resolution with trajectory recovery using mixed-integer programming","authors":"Fernando Dias, David Rey","doi":"10.1007/s10898-024-01393-1","DOIUrl":"https://doi.org/10.1007/s10898-024-01393-1","url":null,"abstract":"<p>To guarantee the safety of flight operations, decision-support systems for air traffic control must be able to improve the usage of airspace capacity and handle increasing demand. This study addresses the aircraft conflict avoidance and trajectory recovery problem. The problem of finding the least deviation conflict-free aircraft trajectories that guarantee the return to a target waypoint is highly complex due to the nature of the nonlinear trajectories that are sought. We present a two-stage iterative algorithm that first solves initial conflicts by manipulating their speed and heading control and then identifying each aircraft’s optimal time to recover its trajectory towards their nominal one. We extend existing mixed-integer programming formulations by modelling speed and heading control as continuous variables while recovery time is treated as a discrete variable. We develop a novel iterative approach which shows that the trajectory recovery costs can be anticipated by inducing avoidance trajectories with higher deviation, therefore obtaining earlier recovery time within a few iterations. Numerical results on benchmark conflict resolution problems show that this approach can solve instances with up to 30 aircraft within 10 min.</p>","PeriodicalId":15961,"journal":{"name":"Journal of Global Optimization","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142248768","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-13DOI: 10.1007/s10898-024-01428-7
Shiming Li, Wei Yu, Zhaohui Liu
The k-path partition problem (kPP), defined on a graph (G=(V,E)), is a well-known NP-hard problem when (kge 3). The goal of the kPP is to find a minimum collection of vertex-disjoint paths to cover all the vertices in G such that the number of vertices on each path is no more than k. In this paper, we give two approximation algorithms for the kPP. The first one, called Algorithm 1, uses an algorithm for the (0,1)-weighted maximum traveling salesman problem as a subroutine. When G is undirected, the approximation ratio of Algorithm 1 is (frac{k+12}{7} -frac{6}{7k} ), which improves on the previous best-known approximation algorithm for every (kge 7). When G is directed, Algorithm 1 is a (left( frac{k+6}{4} -frac{3}{4k}right) )-approximation algorithm, which improves the existing best available approximation algorithm for every (kge 10). Our second algorithm, i.e. Algorithm 2, is a local search algorithm tailored for the kPP in undirected graphs with small k. Algorithm 2 improves on the approximation ratios of the best available algorithm for every (k=4,5,6). Combined with Algorithms 1 and 2, we have improved the approximation ratio for the kPP in undirected graphs for each (kge 4) as well as the approximation ratio for the kPP in directed graphs for each (kge 10). As for the negative side, we show that for any (epsilon >0) it is NP-hard to approximate the kPP (with k being part of the input) within the ratio (O(k^{1-epsilon })), which implies that Algorithm 1 is asymptotically optimal.
k 路径分割问题(kPP)定义在图(G=(V,E))上,是一个众所周知的 NP 难问题(当 (kge 3) 时)。kPP 的目标是找到覆盖 G 中所有顶点的顶点不相交路径的最小集合,使得每条路径上的顶点数不超过 k。第一种算法称为算法 1,它使用 (0,1)-weighted maximum traveling salesman 问题的算法作为子程序。当G是无向的,算法1的近似率是(frac{k+12}{7} -frac{6}{7k} ),这改进了之前已知的每(kge 7)的近似算法。当G是有向的,算法1是一个((left( frac{k+6}{4} -frac{3}{4k}right) )近似算法,它改进了现有的每一个(kge 10)的最佳近似算法。我们的第二种算法,即算法 2,是一种局部搜索算法,专为 k 较小的无向图中的 kPP 量身定制。算法 2 提高了现有最佳算法对每(k=4,5,6)个图的近似率。结合算法1和算法2,我们改进了无向图中每一个(k=4,5,6)的kPP近似率,以及有向图中每一个(k=10)的kPP近似率。至于反面,我们证明了对于任意(epsilon >0)来说,在比率(O(k^{1-epsilon }))内逼近kPP(k是输入的一部分)是NP-hard的,这意味着算法1是渐进最优的。
{"title":"Improved approximation algorithms for the k-path partition problem","authors":"Shiming Li, Wei Yu, Zhaohui Liu","doi":"10.1007/s10898-024-01428-7","DOIUrl":"https://doi.org/10.1007/s10898-024-01428-7","url":null,"abstract":"<p>The <i>k</i>-path partition problem (kPP), defined on a graph <span>(G=(V,E))</span>, is a well-known NP-hard problem when <span>(kge 3)</span>. The goal of the kPP is to find a minimum collection of vertex-disjoint paths to cover all the vertices in <i>G</i> such that the number of vertices on each path is no more than <i>k</i>. In this paper, we give two approximation algorithms for the kPP. The first one, called Algorithm 1, uses an algorithm for the (0,1)-weighted maximum traveling salesman problem as a subroutine. When <i>G</i> is undirected, the approximation ratio of Algorithm 1 is <span>(frac{k+12}{7} -frac{6}{7k} )</span>, which improves on the previous best-known approximation algorithm for every <span>(kge 7)</span>. When <i>G</i> is directed, Algorithm 1 is a <span>(left( frac{k+6}{4} -frac{3}{4k}right) )</span>-approximation algorithm, which improves the existing best available approximation algorithm for every <span>(kge 10)</span>. Our second algorithm, i.e. Algorithm 2, is a local search algorithm tailored for the kPP in undirected graphs with small <i>k</i>. Algorithm 2 improves on the approximation ratios of the best available algorithm for every <span>(k=4,5,6)</span>. Combined with Algorithms 1 and 2, we have improved the approximation ratio for the kPP in undirected graphs for each <span>(kge 4)</span> as well as the approximation ratio for the kPP in directed graphs for each <span>(kge 10)</span>. As for the negative side, we show that for any <span>(epsilon >0)</span> it is NP-hard to approximate the kPP (with <i>k</i> being part of the input) within the ratio <span>(O(k^{1-epsilon }))</span>, which implies that Algorithm 1 is asymptotically optimal.</p>","PeriodicalId":15961,"journal":{"name":"Journal of Global Optimization","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142175849","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-12DOI: 10.1007/s10898-024-01429-6
A. Ghaffari-Hadigheh, L. Sinjorgo, R. Sotirov
We propose a random coordinate descent algorithm for optimizing a non-convex objective function subject to one linear constraint and simple bounds on the variables. Although it is common use to update only two random coordinates simultaneously in each iteration of a coordinate descent algorithm, our algorithm allows updating arbitrary number of coordinates. We provide a proof of convergence of the algorithm. The convergence rate of the algorithm improves when we update more coordinates per iteration. Numerical experiments on large scale instances of different optimization problems show the benefit of updating many coordinates simultaneously.
{"title":"On convergence of a q-random coordinate constrained algorithm for non-convex problems","authors":"A. Ghaffari-Hadigheh, L. Sinjorgo, R. Sotirov","doi":"10.1007/s10898-024-01429-6","DOIUrl":"https://doi.org/10.1007/s10898-024-01429-6","url":null,"abstract":"<p>We propose a random coordinate descent algorithm for optimizing a non-convex objective function subject to one linear constraint and simple bounds on the variables. Although it is common use to update only two random coordinates simultaneously in each iteration of a coordinate descent algorithm, our algorithm allows updating arbitrary number of coordinates. We provide a proof of convergence of the algorithm. The convergence rate of the algorithm improves when we update more coordinates per iteration. Numerical experiments on large scale instances of different optimization problems show the benefit of updating many coordinates simultaneously.</p>","PeriodicalId":15961,"journal":{"name":"Journal of Global Optimization","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2024-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142175853","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-12DOI: 10.1007/s10898-024-01430-z
Chunhua Tang, Shuangyao Zhao, Han Su, Binbin Chen
Manufacturing service composition (MSC) is a core technology in cloud manufacturing (CMfg), which has been intensively studied to find an optimal composite service with the best quality of service (QoS). With the continuous expansion of CMfg platforms, the difficulty of MSC is gradually increasing. Large-scale platforms have put forward higher requirements for combination efficiency, and its open and dynamic environment makes service QoS exhibit strong uncertainty, leading to reliability issues of MSC. Meanwhile, the increased number of services and users makes it necessary for the platform to consider the sustainability issue, including economic, environmental, and social aspects, based on an operations management perspective. However, current studies only consider part of efficiency, reliability, and sustainability as optimization objectives in MSC allocation models, and do not take them into account simultaneously in an integrated manner. Therefore, this study proposes a two-stage method integrating clustering and multi-objective optimization for reliable and sustainable MSC allocation. Specifically, in the first stage, the K-means clustering technique and the QoS stability-based service pruning mechanism are integrated into the service clustering process to improve the reliability of candidate services and reduce the search space of combinations. In the second stage, a multi-objective optimization model with maximizing QoS and sustainability is proposed to find the optimal MSC, and the fast non-dominated sorting genetic algorithm is adopted to solve the model. Finally, a case study of the actual production of a customized automated guided vehicle verifies the effectiveness of the proposed two-stage method.
{"title":"A QoS and sustainability-driven two-stage service composition method in cloud manufacturing: combining clustering and bi-objective optimization","authors":"Chunhua Tang, Shuangyao Zhao, Han Su, Binbin Chen","doi":"10.1007/s10898-024-01430-z","DOIUrl":"https://doi.org/10.1007/s10898-024-01430-z","url":null,"abstract":"<p>Manufacturing service composition (MSC) is a core technology in cloud manufacturing (CMfg), which has been intensively studied to find an optimal composite service with the best quality of service (QoS). With the continuous expansion of CMfg platforms, the difficulty of MSC is gradually increasing. Large-scale platforms have put forward higher requirements for combination efficiency, and its open and dynamic environment makes service QoS exhibit strong uncertainty, leading to reliability issues of MSC. Meanwhile, the increased number of services and users makes it necessary for the platform to consider the sustainability issue, including economic, environmental, and social aspects, based on an operations management perspective. However, current studies only consider part of efficiency, reliability, and sustainability as optimization objectives in MSC allocation models, and do not take them into account simultaneously in an integrated manner. Therefore, this study proposes a two-stage method integrating clustering and multi-objective optimization for reliable and sustainable MSC allocation. Specifically, in the first stage, the <i>K</i>-means clustering technique and the QoS stability-based service pruning mechanism are integrated into the service clustering process to improve the reliability of candidate services and reduce the search space of combinations. In the second stage, a multi-objective optimization model with maximizing QoS and sustainability is proposed to find the optimal MSC, and the fast non-dominated sorting genetic algorithm is adopted to solve the model. Finally, a case study of the actual production of a customized automated guided vehicle verifies the effectiveness of the proposed two-stage method.</p>","PeriodicalId":15961,"journal":{"name":"Journal of Global Optimization","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2024-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142175850","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-03DOI: 10.1007/s10898-024-01424-x
Yijiang Li, Santanu S. Dey, Nikolaos V. Sahinidis
Gas networks are used to transport natural gas, which is an important resource for both residential and industrial customers throughout the world. The gas network design problem is generally modelled as a nonconvex mixed-integer nonlinear integer programming problem (MINLP). The challenges of solving the resulting MINLP arise due to the nonlinearity and nonconvexity. In this paper, we propose a framework to study the “design variant” of the problem in which the variables are the diameter choices of the pipes, the flows, the potentials, and the states of various network components. We utilize a nested loop that includes a two-stage procedure that involves a convex reformulation of the original problem in the inner loop and an efficient enumeration scheme in the outer loop. We conduct experiments on benchmark networks to validate and analyze the performance of our framework.
{"title":"A reformulation-enumeration MINLP algorithm for gas network design","authors":"Yijiang Li, Santanu S. Dey, Nikolaos V. Sahinidis","doi":"10.1007/s10898-024-01424-x","DOIUrl":"https://doi.org/10.1007/s10898-024-01424-x","url":null,"abstract":"<p>Gas networks are used to transport natural gas, which is an important resource for both residential and industrial customers throughout the world. The gas network design problem is generally modelled as a nonconvex mixed-integer nonlinear integer programming problem (MINLP). The challenges of solving the resulting MINLP arise due to the nonlinearity and nonconvexity. In this paper, we propose a framework to study the “design variant” of the problem in which the variables are the diameter choices of the pipes, the flows, the potentials, and the states of various network components. We utilize a nested loop that includes a two-stage procedure that involves a convex reformulation of the original problem in the inner loop and an efficient enumeration scheme in the outer loop. We conduct experiments on benchmark networks to validate and analyze the performance of our framework.\u0000</p>","PeriodicalId":15961,"journal":{"name":"Journal of Global Optimization","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2024-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142175851","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-14DOI: 10.1007/s10898-024-01406-z
Suning Gong, Qingqin Nong, Yue Wang, Dingzhu Du
In this paper, we study the budget-constrained profit maximization problem with expensive seed endorsement, a derivation of the well-studied influence maximization and profit maximization in social networks. While existing research requires the non-negativity of the objective profit function, this paper considers real-world scenarios where costs may surpass revenue. Specifically, our problem can be regarded as maximizing the difference between a non-negative submodular function and a non-negative modular function under a knapsack constraint, allowing for negative differences. To tackle this challenge, we propose two algorithms. Firstly, we employ a twin greedy and enumeration technique to design a polynomial-time algorithm with a quarter weak approximation ratio, providing a balance between computational efficiency and solution quality. Then, we incorporate a threshold decreasing technique to enhance the time complexity of the first algorithm, yielding an improved computational efficiency while maintaining a reasonable level of solution accuracy. To our knowledge, this is the first paper to study the profit maximization beyond non-negativity and to propose polynomial-time algorithms with a constant bicriteria approximation ratio.
{"title":"Budget-constrained profit maximization without non-negative objective assumption in social networks","authors":"Suning Gong, Qingqin Nong, Yue Wang, Dingzhu Du","doi":"10.1007/s10898-024-01406-z","DOIUrl":"https://doi.org/10.1007/s10898-024-01406-z","url":null,"abstract":"<p>In this paper, we study the budget-constrained profit maximization problem with expensive seed endorsement, a derivation of the well-studied influence maximization and profit maximization in social networks. While existing research requires the non-negativity of the objective profit function, this paper considers real-world scenarios where costs may surpass revenue. Specifically, our problem can be regarded as maximizing the difference between a non-negative submodular function and a non-negative modular function under a knapsack constraint, allowing for negative differences. To tackle this challenge, we propose two algorithms. Firstly, we employ a twin greedy and enumeration technique to design a polynomial-time algorithm with a quarter weak approximation ratio, providing a balance between computational efficiency and solution quality. Then, we incorporate a threshold decreasing technique to enhance the time complexity of the first algorithm, yielding an improved computational efficiency while maintaining a reasonable level of solution accuracy. To our knowledge, this is the first paper to study the profit maximization beyond non-negativity and to propose polynomial-time algorithms with a constant bicriteria approximation ratio.</p>","PeriodicalId":15961,"journal":{"name":"Journal of Global Optimization","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2024-08-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142175852","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-12DOI: 10.1007/s10898-024-01423-y
Joaquim Júdice, Valentina Sessa, Masao Fukushima
We introduce a new sequential algorithm for the Standard Quadratic Programming Problem (StQP), which exploits a formulation of StQP as a Linear Program with Linear Complementarity Constraints (LPLCC). The algorithm is finite and guarantees at least in theory a (delta )-approximate global minimum for an arbitrary small (delta ), which is a global minimum in practice. The sequential algorithm has two phases. In Phase 1, Stationary Points (SP) with strictly decreasing objective function values are computed. Phase 2 is designed for giving a certificate of global optimality for the last SP computed in Phase 1. Two different Nonlinear Programming Formulations for LPLCC are proposed for each one of these phases, which are solved by efficient enumerative algorithms. New procedures for computing a lower bound for StQP are also proposed, which are easy to implement and give tight bounds in general. Computational experiments with a number of test problems from known sources indicate that the two-phase sequential algorithm is, in general, efficient in practice. Furthermore, the algorithm seems to be an efficient way to study the copositivity of a matrix by exploiting an StQP with this matrix.
{"title":"A two-phase sequential algorithm for global optimization of the standard quadratic programming problem","authors":"Joaquim Júdice, Valentina Sessa, Masao Fukushima","doi":"10.1007/s10898-024-01423-y","DOIUrl":"https://doi.org/10.1007/s10898-024-01423-y","url":null,"abstract":"<p>We introduce a new sequential algorithm for the Standard Quadratic Programming Problem (StQP), which exploits a formulation of StQP as a Linear Program with Linear Complementarity Constraints (LPLCC). The algorithm is finite and guarantees at least in theory a <span>(delta )</span>-approximate global minimum for an arbitrary small <span>(delta )</span>, which is a global minimum in practice. The sequential algorithm has two phases. In Phase 1, Stationary Points (SP) with strictly decreasing objective function values are computed. Phase 2 is designed for giving a certificate of global optimality for the last SP computed in Phase 1. Two different Nonlinear Programming Formulations for LPLCC are proposed for each one of these phases, which are solved by efficient enumerative algorithms. New procedures for computing a lower bound for StQP are also proposed, which are easy to implement and give tight bounds in general. Computational experiments with a number of test problems from known sources indicate that the two-phase sequential algorithm is, in general, efficient in practice. Furthermore, the algorithm seems to be an efficient way to study the copositivity of a matrix by exploiting an StQP with this matrix.</p>","PeriodicalId":15961,"journal":{"name":"Journal of Global Optimization","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2024-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141942916","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-09DOI: 10.1007/s10898-024-01426-9
T. Giovannelli, O. Sohab, L. N. Vicente
We are interested in assessing the use of neural networks as surrogate models to approximate and minimize objective functions in optimization problems. While neural networks are widely used for machine learning tasks such as classification and regression, their application in solving optimization problems has been limited. Our study begins by determining the best activation function for approximating the objective functions of popular nonlinear optimization test problems, and the evidence provided shows that ReLU and SiLU exhibit the best performance on both training and testing data. We then analyze the accuracy of function value, gradient, and Hessian approximations for such objective functions obtained through interpolation/regression models and neural networks. When compared to interpolation/regression models, neural networks can deliver competitive zero- and first-order approximations (at a high training cost) but underperform on second-order approximation. However, it is shown that combining a neural net activation function with the natural basis for quadratic interpolation/regression can waive the necessity of including cross terms in the natural basis, leading to models with fewer parameters to determine. Lastly, we provide evidence that the performance of a state-of-the-art derivative-free optimization algorithm can hardly be improved when the gradient of an objective function is approximated using any of the surrogate models considered, including neural networks.
我们有兴趣评估在优化问题中使用神经网络作为近似和最小化目标函数的代理模型。虽然神经网络被广泛用于分类和回归等机器学习任务,但其在解决优化问题方面的应用却很有限。我们的研究首先确定了近似常用非线性优化测试问题目标函数的最佳激活函数,所提供的证据表明,ReLU 和 SiLU 在训练和测试数据上都表现出最佳性能。然后,我们分析了通过插值/回归模型和神经网络获得的此类目标函数的函数值、梯度和赫塞斯近似值的准确性。与插值/回归模型相比,神经网络可以提供有竞争力的零阶和一阶近似(训练成本较高),但在二阶近似方面表现不佳。不过,研究表明,将神经网络激活函数与二次插值/回归的自然基相结合,可以免除在自然基中加入交叉项的必要性,从而减少模型需要确定的参数。最后,我们提供的证据表明,当使用包括神经网络在内的任何代用模型对目标函数梯度进行逼近时,最先进的无导数优化算法的性能很难得到改善。
{"title":"The limitation of neural nets for approximation and optimization","authors":"T. Giovannelli, O. Sohab, L. N. Vicente","doi":"10.1007/s10898-024-01426-9","DOIUrl":"https://doi.org/10.1007/s10898-024-01426-9","url":null,"abstract":"<p>We are interested in assessing the use of neural networks as surrogate models to approximate and minimize objective functions in optimization problems. While neural networks are widely used for machine learning tasks such as classification and regression, their application in solving optimization problems has been limited. Our study begins by determining the best activation function for approximating the objective functions of popular nonlinear optimization test problems, and the evidence provided shows that ReLU and SiLU exhibit the best performance on both training and testing data. We then analyze the accuracy of function value, gradient, and Hessian approximations for such objective functions obtained through interpolation/regression models and neural networks. When compared to interpolation/regression models, neural networks can deliver competitive zero- and first-order approximations (at a high training cost) but underperform on second-order approximation. However, it is shown that combining a neural net activation function with the natural basis for quadratic interpolation/regression can waive the necessity of including cross terms in the natural basis, leading to models with fewer parameters to determine. Lastly, we provide evidence that the performance of a state-of-the-art derivative-free optimization algorithm can hardly be improved when the gradient of an objective function is approximated using any of the surrogate models considered, including neural networks.</p>","PeriodicalId":15961,"journal":{"name":"Journal of Global Optimization","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2024-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142175763","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-08DOI: 10.1007/s10898-024-01398-w
Pham Ngoc Anh
{"title":"Relaxed projection methods for solving variational inequality problems","authors":"Pham Ngoc Anh","doi":"10.1007/s10898-024-01398-w","DOIUrl":"https://doi.org/10.1007/s10898-024-01398-w","url":null,"abstract":"","PeriodicalId":15961,"journal":{"name":"Journal of Global Optimization","volume":null,"pages":null},"PeriodicalIF":1.8,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141925708","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}