Position-based acoustic visual servo control for docking of autonomous underwater vehicle using deep reinforcement learning

Abstract

In this paper, a position-based acoustic visual servo control scheme is proposed to achieve efficient docking for underactuated autonomous underwater vehicle (AUV). To address the issue that docking station can hardly be kept in the limited field of onboard imaging sonar using conventional controllers, deep reinforcement learning (DRL) is proposed to learn an optimal control strategy to achieve both field control and precise docking. First, a visualized underwater docking environment is developed based on robotic operation system (ROS) and Gazebo platform, and imaging sonar is modeled to simulate acoustic image. Subsequently, a deep neural network-inspired detector and a state-of-the-art feature tracker are combined as the perceptual header of docking controller which transforms the output of imaging sonar to the input of control agent rapidly and precisely. Furthermore, cost terms of DRL-based control agents are further designed by incorporating two nonlinear functions with different gradients, yet the change rates of received rewards from the state observation are not same in different situations. In this case, the control agents are enabled to learn the strategy to eliminate offset and keep depth of AUV. Moreover, feature of docking station will also be well kept in acoustic image due to the rapid increasing punishment of great bearing angle when AUV is in high maneuvering. Furthermore, evaluation results and comparative experiments are presented to verify feasibility and efficiency of the proposed servo control scheme using deep reinforcement learning.