Joint UAV Trajectory and RadCom Task Schedule for IVNs: A Game-Embedding Multi-Agent Deep Reinforcement Learning Approach

Abstract

Integrated sensing and communication (ISAC) technology has been envisioned to revolutionize the future intelligent vehicle networks (IVNs). Recently, due to the mobility and flexible deployment, unmanned aerial vehicle (UAV) has been regarded as a promising aerial ISAC platform in future IVNs. In this paper, comprehensively considering all the performance on the throughput, sensing accuracy, and sensing rate, we propose a Multi-Agent joint Trajectory control and RadCom task schedule (MA-TRC) scheme for the ISAC-UAV assisted IVN. To make UAVs achieve the optimal decisions autonomously and adaptively, we propose a Game-Embedding Multi-Agent Deep Reinforcement Learning (GE-MADRL) approach. Specifically, considering the complex action space with both discrete and continuous decision variables of UAVs, we develop a multi-agent Parametrized deep Q-network (MAPDQN) based solution, which can help UAVs learn the dynamic and uncertain environment to adaptively obtain the MA-TRC scheme. Furthermore, since UAVs work in a distributed decision making manner, the potential conflicting decisions will impact the network performance. To avoid the decision conflicts among UAVs during the network parameter training, a distributed two-stage Game method is designed as an action adjuster embedded in MAPDQN, by which the learning convergence performance will be further improved and the strategy conflicts can be avoided.