Computational Optimization and Applications最新文献

This paper develops an iterative algorithm to solve nonsmooth nonconvex optimization problems on complete Riemannian manifolds. The algorithm is based on the combination of the well known trust region and bundle methods. According to the process of the most bundle methods, the objective function is approximated by a piecewise linear working model which is updated by adding cutting planes at unsuccessful trial steps. Then at each iteration, by solving a subproblem that employs the working model in the objective function subject to the trust region, a candidate descent direction is obtained. We study the algorithm from both theoretical and practical points of view and its global convergence is verified to stationary points for locally Lipschitz functions. Moreover, in order to demonstrate the reliability and efficiency, a MATLAB implementation of the proposed algorithm is prepared and results of numerical experiments are reported.

We introduce SPIRAL, a SuPerlinearly convergent Incremental pRoximal ALgorithm, for solving nonconvex regularized finite sum problems under a relative smoothness assumption. Each iteration of SPIRAL consists of an inner and an outer loop. It combines incremental gradient updates with a linesearch that has the remarkable property of never being triggered asymptotically, leading to superlinear convergence under mild assumptions at the limit point. Simulation results with L-BFGS directions on different convex, nonconvex, and non-Lipschitz differentiable problems show that our algorithm, as well as its adaptive variant, are competitive to the state of the art.

In this work, we introduce new direct-search schemes for the solution of bilevel optimization (BO) problems. Our methods rely on a fixed accuracy blackbox oracle for the lower-level problem, and deal both with smooth and potentially nonsmooth true objectives. We thus analyze for the first time in the literature direct-search schemes in these settings, giving convergence guarantees to approximate stationary points, as well as complexity bounds in the smooth case. We also propose the first adaptation of mesh adaptive direct-search schemes for BO. Some preliminary numerical results on a standard set of bilevel optimization problems show the effectiveness of our new approaches.

Flag manifolds, sets of nested sequences of linear subspaces with fixed dimensions, are rising in numerical analysis and statistics. The current optimization algorithms on flag manifolds are based on the exponential map and parallel transport, which are expensive to compute. In this paper we propose practical optimization methods on flag manifolds without the exponential map and parallel transport. Observing that flag manifolds have a similar homogeneous structure with Grassmann and Stiefel manifolds, we generalize some typical retractions and vector transports to flag manifolds, including the Cayley-type retraction and vector transport, the QR-based and polar-based retractions, the projection-type vector transport and the projection of the differentiated polar-based retraction as a vector transport. Theoretical properties and efficient implementations of the proposed retractions and vector transports are discussed. Then we establish Riemannian gradient and Riemannian conjugate gradient algorithms based on these retractions and vector transports. Numerical results on the problem of nonlinear eigenflags demonstrate that our algorithms have a great advantage in efficiency over the existing ones.

This is Part II of a study on mixed-integer programming (MIP) relaxation techniques for the solution of non-convex mixed-integer quadratically constrained quadratic programs (MIQCQPs). We set the focus on MIP relaxation methods for non-convex continuous variable products where both variables are bounded and extend the well-known MIP relaxation normalized multiparametric disaggregation technique(NMDT), applying a sophisticated discretization to both variables. We refer to this approach as doubly discretized normalized multiparametric disaggregation technique (D-NMDT). In a comprehensive theoretical analysis, we underline the theoretical advantages of the enhanced method D-NMDT compared to NMDT. Furthermore, we perform a broad computational study to demonstrate its effectiveness in terms of producing tight dual bounds for MIQCQPs. Finally, we compare D-NMDT to the separable MIP relaxations from Part I and a state-of-the-art MIQCQP solver.

We study a novel inertial proximal-gradient method for composite optimization. The proposed method alternates between a variable metric proximal-gradient iteration with momentum and an Armijo-like linesearch based on the sufficient decrease of a suitable merit function. The linesearch procedure allows for a major flexibility on the choice of the algorithm parameters. We prove the convergence of the iterates sequence towards a stationary point of the problem, in a Kurdyka–Łojasiewicz framework. Numerical experiments on a variety of convex and nonconvex problems highlight the superiority of our proposal with respect to several standard methods, especially when the inertial parameter is selected by mimicking the Conjugate Gradient updating rule.

The recent advance of algorithms for nonlinear semidefinite optimization problems (NSDPs) is remarkable. Yamashita et al. first proposed a primal–dual interior point method (PDIPM) for solving NSDPs using the family of Monteiro–Zhang (MZ) search directions. Since then, various kinds of PDIPMs have been proposed for NSDPs, but, as far as we know, all of them are based on the MZ family. In this paper, we present a PDIPM equipped with the family of Monteiro–Tsuchiya (MT) directions, which were originally devised for solving linear semidefinite optimization problems as were the MZ family. We further prove local superlinear convergence to a Karush–Kuhn–Tucker point of the NSDP in the presence of certain general assumptions on scaling matrices, which are used in producing the MT search directions. Finally, we conduct numerical experiments to compare the efficiency among members of the MT family.

We propose Riemannian preconditioned algorithms for the tensor completion problem via tensor ring decomposition. A new Riemannian metric is developed on the product space of the mode-2 unfolding matrices of the core tensors in tensor ring decomposition. The construction of this metric aims to approximate the Hessian of the cost function by its diagonal blocks, paving the way for various Riemannian optimization methods. Specifically, we propose the Riemannian gradient descent and Riemannian conjugate gradient algorithms. We prove that both algorithms globally converge to a stationary point. In the implementation, we exploit the tensor structure and adopt an economical procedure to avoid large matrix formulation and computation in gradients, which significantly reduces the computational cost. Numerical experiments on various synthetic and real-world datasets—movie ratings, hyperspectral images, and high-dimensional functions—suggest that the proposed algorithms have better or favorably comparable performance to other candidates.

Gradient-based methods have been highly successful for solving a variety of both unconstrained and constrained nonlinear optimization problems. In real-world applications, such as optimal control or machine learning, the necessary function and derivative information may be corrupted by noise, however. Sun and Nocedal have recently proposed a remedy for smooth unconstrained problems by means of a stabilization of the acceptance criterion for computed iterates, which leads to convergence of the iterates of a trust-region method to a region of criticality (Sun and Nocedal in Math Program 66:1–28, 2023. https://doi.org/10.1007/s10107-023-01941-9). We extend their analysis to the successive linear programming algorithm (Byrd et al. in Math Program 100(1):27–48, 2003. https://doi.org/10.1007/s10107-003-0485-4, SIAM J Optim 16(2):471–489, 2005. https://doi.org/10.1137/S1052623403426532) for unconstrained optimization problems with objectives that can be characterized as the composition of a polyhedral function with a smooth function, where the latter and its gradient may be corrupted by noise. This gives the flexibility to cover, for example, (sub)problems arising in image reconstruction or constrained optimization algorithms. We provide computational examples that illustrate the findings and point to possible strategies for practical determination of the stabilization parameter that balances the size of the critical region with a relaxation of the acceptance criterion (or descent property) of the algorithm.

We propose an efficient primal-dual interior-point relaxation algorithm based on a smoothing barrier augmented Lagrangian, called IPRSDP, for solving semidefinite programming problems in this paper. The IPRSDP algorithm has three advantages over classical interior-point methods. Firstly, IPRSDP does not require the iterative points to be positive definite. Consequently, it can easily be combined with the warm-start technique used for solving many combinatorial optimization problems, which require the solutions of a series of semidefinite programming problems. Secondly, the search direction of IPRSDP is symmetric in itself, and hence the symmetrization procedure is not required any more. Thirdly, with the introduction of the smoothing barrier augmented Lagrangian function, IPRSDP can provide the explicit form of the Schur complement matrix. This enables the complexity of forming this matrix in IPRSDP to be comparable to or lower than that of many existing search directions. The global convergence of IPRSDP is established under suitable assumptions. Numerical experiments are made on the SDPLIB set, which demonstrate the efficiency of IPRSDP.