A brief survey of line search methods for optimization problems

Abstract

The line search methods for optimization problems have garnered widespread adoption across various domains and applications, primarily due to their effectiveness in addressing intricate problems. An important component that ensures the success of various iterative algorithms is the search direction ($d_k$) while the step-size (${\alpha }_k$) ensures global convergence in different schemes. While the literature offers general guidelines for line search selection, few studies investigate how specific problem constraints impact the performance of optimization methods. This paper presents a comprehensive survey and classification of line search methods, focusing on their computational efficiency and performance under varied problem constraints. We examine the influence of different line search parameters across standard test functions through extensive numerical tests. Our findings suggest practical guidelines for selecting suitable line search methods based on problem characteristics, offering researchers insights into method suitability, and contributing to the significant practical application of optimization in diverse fields.