Joint Deep Reinforcement Learning and Unfolding for Sensing and Communication Function Selection in Vehicular Networks

Abstract

With the exponential advancement of vehicle networking applications and autonomous driving technology, the demand for efficient and secure autonomous vehicles (AVs) is increasing. AVs require the ability to gather information to navigate complex and ever-changing traffic environments, making effective communication with other vehicles or roadside units (RSUs) crucial for achieving co-awareness. Integrated Sensing and Communication (ISAC) technology emerges as a promising solution for the future of autonomous driving. However, in the dynamic and uncertain real-world road environment, the selection of sensing and communication (SC) functions becomes paramount in enhancing performance. Moreover, ambient noise often disrupts the interaction between vehicles and roadside units, leading to a partial loss of environmental states. To address this challenge, we propose a novel approach for selecting sensing and communication functions, even in the presence of partial loss of environmental states. Specifically, we approximate a partially observable Markov decision process (POMDP) to a complete Markov decision process (MDP) through matrix completion and subsequently utilize deep reinforcement learning (DRL) to solve it. Additionally, we propose a matrix completion algorithm based on the alternating direction method of multipliers (ADMM) with deep unfolding to accurately complete the missing environmental states. Finally, we demonstrate that the proposed method outperforms the other POMDP-based approaches for SC function selection in an ISAC-enabled vehicular network.