Adaptive multimodal control of trans-media vehicle based on deep reinforcement learning

Abstract

To solve the problem that the control system is prone to instability due to the sudden change of physical characteristics, strong interference, and nonlinear in the process of multimodal movement of trans-media vehicle, an adaptive control method combining the Deep Deterministic Policy Gradient (DDPG) and traditional Proportional-Integral-Derivative (PID) controller is proposed in this paper. In this approach, the upper-level DDPG controller continuously monitors the vehicle's state and environmental conditions, dynamically adjusting the PID parameters in real-time. The lower-level PID controller then utilizes these updated parameters to modulate the output thrust of the vehicle's motors, thereby achieving excellent control over the vehicle's entire movement. Firstly, according to the hydrodynamic analysis, the kinematics and dynamics mathematical model of the self-designed trans-media vehicle is constructed. This model includes the multi-stage motion modal process of aerial flight, underwater navigation, and cross-media motion, which is suitable for the simulation and verification of the control method. Then, an adaptive controller called RL-PID combining DDPG and PID is built, so that PID can adjust parameters in real-time according to the changes in the external environment. Finally, after theoretical stability proof, a comparison study is performed across three approaches, namely the novel RL-PID, Fuzzy PID, and PID. The experimental results illustrate the superiority of the proposed approach over the competing ones and the generalization of the proposed approach under different interference.