Performance Improvement of Logistic Regression for Binary Classification by Gauss-Newton Method

Abstract

This paper proposes a new approach to optimizing cost function for binary logistic regression by the Gauss-Newton method. This method was applied to the backpropagation phase as a part of the training process to update the weighted coefficients. To show the performance of the approach, we used two data sets to train the logistic regression model for binary classification problems. Our experiment demonstrated that the proposed methods could perform better than gradient descent for both examples, as we expected. Furthermore, the performance of our approach is more advanced than the classical method, either in speed or accuracy.