Target pursuit for multi-AUV system: zero-sum stochastic game with WoLF-PHC assisted

Abstract

Due to the complexity of the underwater environment and the difficulty of the underwater energy recharging, utilizing multiple autonomous underwater vehicles (AUVs) to pursue the invading vehicle is a challenging project. This paper focuses on devising the rational and energy-efficient pursuit motion for a multi-AUV system in an unknown three-dimensional environment. Firstly, the pursuit system model is constructed on the two-player zero-sum stochastic game (ZSSG) framework. This framework enables the fictitious play on the behaviors of the invading AUV. Fictitious play involves players updating their strategies by observing and inferring the actions of others under incomplete information. Under this framework, a relay-pursuit mechanism is adopted by the pursuit system to form the action set in an energy-efficient way. Then, to reflect the pursuit goals of capturing the invading vehicle as soon as possible and avoid it from reaching its point of attack, two corresponding pursuit factors are considered in the designed reward function. To enable the pursuit AUVs to navigate in an unknown environment, WoLF-PHC algorithm is introduced and applied to the proposed ZSSG-based model. Finally, simulations demonstrate the effectiveness, the advantages, and the robustness of the proposed approach.