Fast Gradient Algorithm with Dry-like Friction and Nonmonotone Line Search for Nonconvex Optimization Problems

SIAM Journal on Optimization, Volume 34, Issue 3, Page 2557-2587, September 2024.
Abstract. In this paper, we propose a fast gradient algorithm for the problem of minimizing a differentiable (possibly nonconvex) function in Hilbert spaces. We first extend the dry friction property for convex functions to what we call the dry-like friction property in a nonconvex setting, and then employ a line search technique to adaptively update parameters at each iteration. Depending on the choice of parameters, the proposed algorithm exhibits subsequential convergence to a critical point or full sequential convergence to an “approximate” critical point of the objective function. We also establish the full sequential convergence to a critical point under the Kurdyka–Łojasiewicz (KL) property of a merit function. Thanks to the parameters’ flexibility, our algorithm can reduce to a number of existing inertial gradient algorithms with Hessian damping and dry friction. By exploiting variational properties of the Moreau envelope, the proposed algorithm is adapted to address weakly convex nonsmooth optimization problems. In particular, we extend the result on KL exponent for the Moreau envelope of a convex KL function to a broad class of KL functions that are not necessarily convex nor continuous. Simulation results illustrate the efficiency of our algorithm and demonstrate the potential advantages of combining dry-like friction with extrapolation and line search techniques.