Reinforcement-Learning-Based Smart AUV-IoUT Localization in Underwater Acoustic Topology Network

Abstract

In the Internet of Underwater Things, which is entirely composed of autonomous underwater vehicles (AUVs) without fixed beacons, synchronized underwater acoustic (UWA) localization systems are employed. These systems estimate the relative locations of the AUVs, enabling the formation of an optimal network topology. The localization accuracy is affected by the AUV motion, the localization and communication signal (LCS) bandwidth and duration, and the power of LCSs. In this article, a reinforcement learning (RL)-based smart AUV localization scheme is proposed first. By optimizing the localization time windows and the signal weights, the RL-based localization scheme reduces the localization time delay and energy consumption while increasing the localization accuracy without fixed anchor nodes. Furthermore, a double deep Q-network (DDQN)-based hierarchical structure localization scheme is proposed to optimize the allocation of the substrategies for the follower AUVs, aiming to reduce the localization error and the energy consumption. Computational complexity and Cramér-Rao lower bounds (CRBs) are analyzed to evaluate the optimal localization performance. Simulation results show that the proposed schemes improve the localization accuracy, and reduce the time delay and the energy consumption compared with the benchmarks.