A Modified Contracting BFGS Update for Unconstrained Optimization

Abstract

The Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm is one of the most popular algorithms for solving unconstrained problems. Recently, it has been widely adopted in large-scale optimization problems. However, its use of the Hessian approximation $B_{k}$ is very likely to become ill-conditioned, resulting in an inaccurate search direction. The contracting BFGS algorithm not only retains the positive definiteness of the Hessian approximation $B_{k}$ and the quadratic termination property but also contracts the distribution of the eigenvalues of $B_{k}$ in some sense. However, the argument is not sufficient concerning the improvement of the search direction's accuracy when $B_{k}$ is ill-conditioned. In this paper, we present a modification of the contracting BFGS algorithm for unconstrained optimization. In our algorithm, instead of using a constant contracting factor $c$, we select a different $c_{k}$ in each step. Our algorithm preserves the positive definiteness of $B_{k}$ and the quadratic termination property. Moreover, by choosing a different contracting factor in each iteration, we prove the existence of the ‘best’ $c_{k}$ that minimizes the spectral condition number of $B_{k+1}$. We present a method to find such $c_{k}$ based on the matrix rank-1 perturbation theory and the eigenvalue optimization. Finally, numerical experiments are presented to verify both the convergence property and the improvement of the sensitiveness of the linear systems used to solve for search directions.