Underwater Target Tracking Based on Interrupted Software-Defined Multi-AUV Reinforcement Learning: A Multi-AUV Time-Saving MARL Approach

Abstract

With the rapid development of underwater materials technology and underwater robot technology, human exploitation of marine resources has been increasingly advanced, which has given rise to various application scenarios for Autonomous Underwater Vehicle (AUV) cluster networks, such as cooperative data collection and target tracking. In this paper, we aim to explore how to utilize networking and swarm intelligence to improve the AUV cluster network’s target tracking performance in a time-saving manner. Specifically, on account of our previous work, we introduce an underwater interrupted mechanism and propose an Interrupted Software-Defined Multi-AUV Reinforcement Learning (ISD-MARL) architecture. For MARL algorithm in ISD-MARL, we propose a time-saving MARL algorithm, S-MADDPG, integrating our proposed action optimization model and action network loss function, to accelerate the convergence of the MARL algorithm. Furthermore, to further improve the AUV cluster network’s path planning performance during the target tracking, we propose an Interrupted Tracking Path Planning Scheme (ITPPS) for the AUV cluster network based on the proposed ISD-MARL and S-MADDPG. The evaluation results showcase that our proposed scheme can effectively plan the underwater target tracking path for the AUV cluster network in a shorter time and outperform various mainstream strategies in terms of convergence speed and training time, etc.