A Reinforcement Learning Method for Quadrotor Attitude Control Based on Expert Information

Abstract

In this paper, a model-free reinforcement learning(RL) method of training a nonlinear attitude controller of a quadrotor is proposed. For the problem that the attitude controller is uncontrolled when trained by RL directly, the proposed method utilizes an expert to provide the prior information, i.e. the action’s judgement and suggestion, to guide the updating process. For the problem that the policy falls in local optima by the limitation of the expert, the proposed method maximize the entropy of the strategy to increase the exploratory behavior of the nonlinear attitude controller approximator. Furthermore, We employ the Proximal policy optimization algorithm (PPO) as the RL model and PID algorithm as the expert model to approach an exact attitude controller of a quadrotor based on the proposed method. Finally, the simulations experiments has been conducted to verify that our proposed method can train a true nonlinear attitude controller which has a better performance than the expert.