Proximal Policy-Optimized Regularized Least Squares Algorithm for Noise-Resilient Motion Prediction of UMVs

Abstract

To enhance the accuracy of motion prediction in unmanned marine vehicles (UMVs), an innovative proximal policy-optimized regularized least squares (PPO-RLS) algorithm is proposed in this article. This article begins by developing a dynamics model for UMVs that incorporates viscous damping and external forces to minimize modeling errors. However, this model does not account for data noise, making accurate parameter identification difficult when using traditional least squares (LS) algorithms. To overcome this limitation, the PPO-RLS algorithm is proposed, incorporating a regularization term within the LS framework and utilizing proximal policy optimization for adaptive regularization term tuning. The performance of the PPO-RLS algorithm is thoroughly evaluated using both simulation data and lake trial data, demonstrating significant improvements over both the traditional LS algorithm and a state-of-the-art algorithm. Specifically, in simulation tests, the PPO-RLS algorithm achieves a notable reduction in root mean square error for surge velocity (8.49E-03 m/s) and heading angle (2.32 $^\circ$ ), markedly outperforming the LS algorithm (2.36E-02 m/s for surge velocity and 4.14 $^\circ$ for heading angle). In addition, the PPO-RLS algorithm displays enhanced stability, as indicated by a more than 50% reduction in condition number (1.46E+04 for PPO-RLS versus 2.89E+06 for LS). These improvements are further validated by lake trial data, confirming the algorithm's advanced motion prediction capabilities with quantitatively lower errors and greater robustness.