Analysis of a new BFGS algorithm and conjugate gradient algorithms and their applications in image restoration and machine learning

Abstract

Renowned for offering a more precise approximation of the objective function, a third-order tensor expansion is deemed superior to the traditional second-order Taylor expansion, a viewpoint supported by various academics. Despite its acknowledged benefits, the adoption of this advanced expansion within the widely utilized quasi-Newton method remains notably rare. This research endeavors to construct update equations for the quasi-Newton method, based on a third-order tensor expansion, and to introduce an innovative quasi-Newton equation. The main contributions of this study include: (i) the development of a unique quasi-Newton equation based on a third-order tensor expansion; (ii) a detailed comparative analysis of the new BFGS quasi-Newton update method versus the traditional BFGS methodologies; (iii) the demonstration of convergence outcomes for the newly developed BFGS quasi-Newton technique; and (iv) the introduction of novel methodologies for conjugate gradients inspired by this distinctive quasi-Newton formula. Through exhaustive numerical experimentation, the algorithms derived from this pioneering quasi-Newton equation have shown superior performance.