Augmented Lagrangian–Based First-Order Methods for Convex-Constrained Programs with Weakly Convex Objective

Abstract

First-order methods (FOMs) have been widely used for solving large-scale problems. A majority of existing works focus on problems without constraint or with simple constraints. Several recent works have studied FOMs for problems with complicated functional constraints. In this paper, we design a novel augmented Lagrangian (AL)–based FOM for solving problems with nonconvex objective and convex constraint functions. The new method follows the framework of the proximal point (PP) method. On approximately solving PP subproblems, it mixes the usage of the inexact AL method (iALM) and the quadratic penalty method, whereas the latter is always fed with estimated multipliers by the iALM. The proposed method achieves the best-known complexity result to produce a near Karush–Kuhn–Tucker (KKT) point. Theoretically, the hybrid method has a lower iteration-complexity requirement than its counterpart that only uses iALM to solve PP subproblems; numerically, it can perform significantly better than a pure-penalty-based method. Numerical experiments are conducted on nonconvex linearly constrained quadratic programs. The numerical results demonstrate the efficiency of the proposed methods over existing ones.