Robust Reinforcement Learning Control Framework for a Quadrotor Unmanned Aerial Vehicle Using Critic Neural Network

Abstract

This article introduces a novel robust reinforcement learning (RL) control scheme for a quadrotor unmanned aerial vehicle (QUAV) under external disturbances and model uncertainties. First, the translational and rotational motions of the QUAV are decoupled and trained separately to mitigate the computational complexity of the controller design and training process. Then, the proximal policy optimization algorithm with a dual-critic structure is proposed to address the overestimation issue and accelerate the convergence speed of RL controllers. Furthermore, a novel reward function and a robust compensator employing a switch value function are proposed to address model uncertainties and external disturbances. At last, simulation results and comparisons demonstrate the effectiveness and robustness of the proposed RL control framework.