An augmented Lagrangian method for nonconvex composite optimization problems with nonlinear constraints

In this paper, we propose an augmented Lagrangian method with Backtracking Line Search for solving nonconvex composite optimization problems including both nonlinear equality and inequality constraints. In case the variable spaces are homogeneous, our setting yields a generic nonlinear mathematical programming model. When some variables belong to the real Hilbert space and others to the integer space, one obtains a nonconvex mixed-integer/-binary nonlinear programming model for which the nonconvexity is not limited to the integrality constraints. Together with the formal proof of its iteration complexity, the proposed algorithm is then numerically evaluated to solve a multi-constrained network design problem. Extensive numerical executions on a set of instances extracted from the SNDlib repository are then performed to study its behavior and performance as well as identify potential improvement of this method. Finally, analysis of the results and their comparison against those obtained when solving its convex relaxation using mixed-integer programming solvers are reported.