Lyapunov-based distributed reinforcement learning control with stability guarantee

Abstract

In this paper, we propose a Lyapunov-based distributed reinforcement control method for nonlinear systems that comprise interacting subsystems; this method provides guaranteed closed-loop stability. Specifically, we conduct stability analysis and provide sufficient conditions that ensure the closed-loop stability of the proposed distributed reinforcement learning control scheme. The Lyapunov-based condition is leveraged to guide the design of a local reinforcement learning controller for each subsystem of the entire system. The local controllers only exchange scalar-valued information during the training phase, yet they do not need to communicate once the training is completed and the controllers are implemented online. The effectiveness and performance of the proposed method are evaluated using a benchmark chemical process that contains two reactors and one separator.