Safe Reinforcement Learning: Optimal Formation Control With Collision Avoidance of Multiple Satellite Systems

Abstract

This article addresses the collision avoidance and formation control problem for multisatellite systems. A novel safe reinforcement learning (RL) algorithm based on an adaptive dynamic programming framework is proposed. The highlights of the algorithm are the adaptive distance-varying learning method to integrate online data with historical data and the usage of the barrier function (BF) to achieve collision avoidance. First, the BF is introduced into the designed cost function such that the multisatellite formation system can achieve obstacle avoidance and guarantee the safety. Next, a safe RL algorithm is developed through the critic network structure. A distance-varying weight is introduced, which combines experience replay samples with extrapolation samples. By minimizing the cost function, the optimal formation control policy can be obtained with an adaptive formation and self-learning ability. Then, the stability and safety of the proposed algorithm are analyzed. Finally, the effectiveness and superiority of the proposed algorithm are verified by numerical simulations.