Situation-aware deep reinforcement learning for autonomous nonlinear mobility control in cyber-physical loitering munition systems

Abstract

With the rapid development of autonomous mobility technologies, drones are now widely used in many applications, including military domain. Particularly in battlefield conditions, designing a deep reinforcement learning (DRL)-based autonomous control algorithm presents significant challenges due to the need for real-time and adjustable nonlinear trajectory planning. Therefore, this paper introduces a novel situation-aware DRL-based autonomous nonlinear drone mobility control algorithm tailored for cyber-physical loitering munition applications. The proposed DRL-based drone mobility control algorithm is crafted with a focus on real-time situation-aware operations, enabling it to navigate through many obstacles encountered on the battlefield efficiently. For efficient observation and intuitive fast understanding of time-varying real-time situations, this paper presents an algorithm that works on a cyber-physical virtual battlefield environment using Unity. In detail, our proposed DRL-based nonlinear drone mobility control algorithm utilizes situation-aware sensing components that are implemented with a Raycast function in Unity virtual scenarios. Based on the gathered situation-aware information, the drone can autonomously and nonlinearly adjust its trajectory during flight. Thus, this approach is obviously beneficial for avoiding obstacles in complex and unpredictable battlefields. Our visualization- based performance evaluation shows that the proposed algorithm outperforms other mobility control algorithms, with an average performance nearly twice as high when the obstacle density is 50%. This superiority is further evidenced by the detailed trajectory planning presented.