Multi-USV Deep Reinforcement Learning for Distributed Cooperative Target Tracking

Abstract

The purpose of this paper is to discuss distributed cooperative target tracking for a multi-unmanned surface vehicle (multi-USV) system. The cooperative target tracking problem is formulated as a multi-USV learning problem. Based on this formulation, a multi-USV distributed cooperative target tracking (MUTT) algorithm is proposed. To avoid the collisions between USVs during the tracking process, an additional safety layer is introduced. Some safety signals are constructed based on USVs' states. By correcting actions through the trained safety layer, USVs can avoid collisions reasonably. Moreover, for the sake of demonstrating the effectiveness of the proposed MUTT algorithm in target tracking, reward functions and mission scenarios are well constructed. Furthermore, a comparison of the MUTT algorithm and Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm is given. The obtained results manifest that the proposed MUTT algorithm provides safe policies for multi-USV cooperative target tracking tasks.