Swarm-intelligence-based value iteration for optimal regulation of continuous-time nonlinear systems

Abstract

In this article, a swarm-intelligence-based value iteration (VI) algorithm is constructed to resolve the optimal control issue for continuous-time (CT) nonlinear systems. By leveraging the evolutionary concept of particle swarm optimization (PSO), the challenge of gradient vanishing is effectively overcome compared to traditional adaptive dynamic programming (ADP). Specifically, a PSO-based action network is implemented to perform policy improvement, eliminating the reliance on gradient information. Furthermore, within the ADP framework, the swarm-intelligence-based VI algorithm for CT systems is developed to address the challenges associated with constraints of initial admissible conditions and the difficulty of selecting probing signals in the traditional policy iteration method. The theoretical analysis is provided to show the convergence of the developed VI algorithm and the stability of the closed-loop system, respectively. Finally, under affine and non-affine backgrounds, two simulations are conducted to demonstrate the effectiveness and optimality of the established swarm-intelligence-based VI scheme for CT systems.