Computational Optimization and Applications最新文献

The Delayed Weighted Gradient Method (DWGM) is a two-step gradient algorithm that is efficient for the minimization of large scale strictly convex quadratic functions. It has orthogonality properties that make it to compete with the Conjugate Gradient (CG) method. Both methods calculate in sequence two step-sizes, CG minimizes the objective function and DWGM the gradient norm, alongside two search directions defined over first order current and previous iteration information. The objective of this work is to accelerate the recently developed extension of DWGM to nonquadratic strongly convex minimization problems. Our idea is to define the step-sizes of DWGM in a unique two dimensional convex quadratic optimization problem, calculating them simultaneously. Convergence of the resulting algorithm is analyzed. Comparative numerical experiments illustrate the effectiveness of our approach.

In this work, we introduce a novel stochastic second-order method, within the framework of a non-monotone trust-region approach, for solving the unconstrained, nonlinear, and non-convex optimization problems arising in the training of deep neural networks. The proposed algorithm makes use of subsampling strategies that yield noisy approximations of the finite sum objective function and its gradient. We introduce an adaptive sample size strategy based on inexpensive additional sampling to control the resulting approximation error. Depending on the estimated progress of the algorithm, this can yield sample size scenarios ranging from mini-batch to full sample functions. We provide convergence analysis for all possible scenarios and show that the proposed method achieves almost sure convergence under standard assumptions for the trust-region framework. We report numerical experiments showing that the proposed algorithm outperforms its state-of-the-art counterpart in deep neural network training for image classification and regression tasks while requiring a significantly smaller number of gradient evaluations.

We introduce a new approach to apply the boosted difference of convex functions algorithm (BDCA) for solving non-convex and non-differentiable problems involving difference of two convex functions (DC functions). Supposing the first DC component differentiable and the second one possibly non-differentiable, the main idea of BDCA is to use the point computed by the subproblem of the DC algorithm (DCA) to define a descent direction of the objective from that point, and then a monotone line search starting from it is performed in order to find a new point which decreases the objective function when compared with the point generated by the subproblem of DCA. This procedure improves the performance of the DCA. However, if the first DC component is non-differentiable, then the direction computed by BDCA can be an ascent direction and a monotone line search cannot be performed. Our approach uses a non-monotone line search in the BDCA (nmBDCA) to enable a possible growth in the objective function values controlled by a parameter. Under suitable assumptions, we show that any cluster point of the sequence generated by the nmBDCA is a critical point of the problem under consideration and provides some iteration-complexity bounds. Furthermore, if the first DC component is differentiable, we present different iteration-complexity bounds and prove the full convergence of the sequence under the Kurdyka–Łojasiewicz property of the objective function. Some numerical experiments show that the nmBDCA outperforms the DCA, such as its monotone version.

In this paper, we propose and analyze an away-step Frank–Wolfe algorithm designed for solving multiobjective optimization problems over polytopes. We prove that each limit point of the sequence generated by the algorithm is a weak Pareto optimal solution. Furthermore, under additional conditions, we show linear convergence of the whole sequence to a Pareto optimal solution. Numerical examples illustrate a promising performance of the proposed algorithm in problems where the multiobjective Frank–Wolfe convergence rate is only sublinear.

Shape optimization methods have been proven useful for identifying interfaces in models governed by partial differential equations. Here we consider a class of shape optimization problems constrained by nonlocal equations which involve interface–dependent kernels. We derive a novel shape derivative associated to the nonlocal system model and solve the problem by established numerical techniques. The code for obtaining the results in this paper is published at (https://github.com/schustermatthias/nlshape).

In this paper, we propose a hybrid inexact regularized Newton and negative curvature method for solving unconstrained nonconvex problems. The descent direction is chosen based on different conditions, either the negative curvature or the inexact regularized direction. In addition, to minimize computational costs while obtaining the negative curvature, we employ a dimensionality reduction strategy to verify if the Hessian matrix exhibits negative curvatures within a three-dimensional subspace. We show that the proposed method can achieve the best-known global iteration complexity if the Hessian of the objective function is Lipschitz continuous on a certain compact set. Two simplified methods for nonconvex and strongly convex problems are analyzed as specific instances of the proposed method. We show that under the local error bound assumption with respect to the gradient, the distance between iterations generated by our proposed method and the local solution set converges to (0) at a superlinear rate. Additionally, for strongly convex problems, the quadratic convergence rate can be achieved. Extensive numerical experiments show the effectiveness of the proposed method.

Chance constraints are a valuable tool for the design of safe decisions in uncertain environments; they are used to model satisfaction of a constraint with a target probability. However, because of possible non-convexity and non-smoothness, optimizing over a chance constrained set is challenging. In this paper, we consider chance constrained programs where the objective function and the constraints are convex with respect to the decision parameter. We establish an exact reformulation of such a problem as a bilevel problem with a convex lower-level. Then we leverage this bilevel formulation to propose a tractable penalty approach, in the setting of finitely supported random variables. The penalized objective is a difference-of-convex function that we minimize with a suitable bundle algorithm. We release an easy-to-use open-source python toolbox implementing the approach, with a special emphasis on fast computational subroutines.

We consider the solution of finite-sum minimization problems, such as those appearing in nonlinear least-squares or general empirical risk minimization problems. We are motivated by problems in which the summand functions are computationally expensive and evaluating all summands on every iteration of an optimization method may be undesirable. We present the idea of stochastic average model (SAM) methods, inspired by stochastic average gradient methods. SAM methods sample component functions on each iteration of a trust-region method according to a discrete probability distribution on component functions; the distribution is designed to minimize an upper bound on the variance of the resulting stochastic model. We present promising numerical results concerning an implemented variant extending the derivative-free model-based trust-region solver POUNDERS, which we name SAM-POUNDERS.

At each iteration of the safeguarded augmented Lagrangian algorithm Algencan, a bound-constrained subproblem consisting of the minimization of the Powell–Hestenes–Rockafellar augmented Lagrangian function is considered, for which an approximate minimizer with tolerance tending to zero is sought. More precisely, a point that satisfies a subproblem first-order necessary optimality condition with tolerance tending to zero is required. In this work, based on the success of scaled stopping criteria in constrained optimization, we propose a scaled stopping criterion for the subproblems of Algencan. The scaling is done with the maximum absolute value of the first-order Lagrange multipliers approximation, whenever it is larger than one. The difference between the convergence theory of the scaled and non-scaled versions of Algencan is discussed and extensive numerical experiments are provided.

We propose a modified BFGS algorithm for multiobjective optimization problems with global convergence, even in the absence of convexity assumptions on the objective functions. Furthermore, we establish a local superlinear rate of convergence of the method under usual conditions. Our approach employs Wolfe step sizes and ensures that the Hessian approximations are updated and corrected at each iteration to address the lack of convexity assumption. Numerical results shows that the introduced modifications preserve the practical efficiency of the BFGS method.