Novel adaptive parameter fractional-order gradient descent learning for stock selection decision support systems

Abstract

Gradient descent methods are widely used as optimization algorithms for updating neural network weights. With advancements in fractional-order calculus, fractional-order gradient descent algorithms have demonstrated superior optimization performance. Nevertheless, existing fractional-order gradient descent algorithms have shortcomings in terms of structural design and theoretical derivation. Specifically, the convergence of fractional-order algorithms in the existing literature relies on the assumed boundedness of network weights. This assumption leads to uncertainty in the optimization results. To address this issue, this paper proposes several adaptive parameter fractional-order gradient descent learning (AP-FOGDL) algorithms based on the Caputo and Riemann–Liouville derivatives. To fully leverage the convergence theorem, an adaptive learning rate is designed by introducing computable upper bounds. The convergence property is then theoretically proven for both derivatives, with and without the adaptive learning rate. Moreover, to enhance prediction accuracy, an amplification factor is employed to increase the adaptive learning rate. Finally, practical applications on a stock selection dataset and a bankruptcy dataset substantiate the feasibility, high accuracy, and strong generalization performance of the proposed algorithms. A comparative study between the proposed methods and other relevant gradient descent methods demonstrates the superiority of the AP-FOGDL algorithms.