A Reinforcement-Learning-Enhanced Spoofing Algorithm for UAV With GPS/INS-Integrated Navigation

Abstract

This article optimizes the covert deception effects on UAV GPS/INS integrated navigation systems by combining spatial information entropy (SIE) and maximum entropy reinforcement learning (MERL) techniques. Specifically, we integrate insights from SIE to meticulously articulate spatial correlations, thereby intricately refining the entropy components within MERL, where this nuanced refinement aims to attain an elevated distribution of navigational spoofing positions. Given that UAV flight control commands are determined exclusively by the current positioning results, regardless of whether the signals are authentic or counterfeit, the navigation deception process satisfies Markov properties. Subsequently, the article establishes theoretical evidence for the Gaussian distribution properties of spoofing positions based on radar Kalman Filter (KF) estimation, and enforces stealth and stability constraints through chi-square distributed random variables. Building on these constraints, a reward function is formulated to jointly optimize deception position concealment, trajectory stability, and successful navigation of the victim UAV to the actual destination. To achieve these objectives, spatial information entropy (SIE) is introduced to model the positional correlations among the deception location, actual destination, and deception destination. Finally, we propose an algorithm based on soft actor-critic (SAC) and SIE, named SIE-SAC, to coordinate the learning process between the deception strategy and the SIE. Without prior knowledge of the UAV's reference trajectory or internal KF parameters, comparative results show that SIE improves deception position concealment. Ablation experiments further validate the constraints' role in stabilizing deceptive trajectories, and the SIE-SAC covert spoofing effect seamlessly extends to three-dimensional scenario.