On the Inversion-Free Newton's Method and Its Applications

Abstract

In this paper, we survey the recent development of inversion-free Newton's method, which directly avoids computing the inversion of Hessian, and demonstrate its applications in estimating parameters of models such as linear and logistic regression. A detailed review of existing methodology is provided, along with comparisons of various competing algorithms. We provide numerical examples that highlight some deficiencies of existing approaches, and demonstrate how the inversion-free methods can improve performance. Motivated by recent works in literature, we provide a unified subsampling framework that can be combined with the inversion-free Newton's method to estimate model parameters including those of linear and logistic regression. Numerical examples are provided for illustration.