Rocket Self-learning Control based on Lightweight Neural Network Architecture Search

Abstract

Aiming at the problem that the traditional control law design process is complex and relies heavily on accurate mathematical models, this paper uses the Deep Deterministic Policy Gradient (DDPG) reinforcement learning to realize the self-learning of continuous motion control law. However, since the performance of the DDPG algorithm depends heavily on the hyper-parameters, there is no clear design basis for the Actor-Critic framework neural network architecture. Considering that the reinforcement learning requires a large amount of computation, the repetitive manual trial and error of hyper-parameters greatly reduces the design efficiency of the algorithm and increases labor costs. On the basis of converting the network architecture design problem into a graph topology generation problem, an automatic search and optimization framework for deep reinforcement learning neural network structure is given in this paper, where the graph topology generation algorithm based on LSTM recurrent neural network, the weight sharing-based lightweight training and evaluation mechanism of deep reinforcement network parameter, and the policy gradient-based learning algorithm of graph topology generator parameter are innovatively combined. Thus, the neural network hyper-parameters in the DDPG algorithm are automatically optimized, and the control law is obtained by self-learning training. Finally, taking rocket vertical recovery control as an ex-ample, the effectiveness of the proposed method is verified.