MVX-ViT: Multimodal Collaborative Perception for 6G V2X Network Management Decisions Using Vision Transformer

Abstract

Advancements in sixth-generation (6G) networks, coupled with the evolution of multimodal sensing in vehicle-to-everything (V2X) networks, have opened avenues for transformative research into multimodal-based artificial intelligence (AI) applications for wireless communication and network management. However, this promising research direction is often constrained by the limited availability of suitable datasets. In response, this paper introduces a comprehensive configurable co-simulation framework that integrates the state-of-the-art CARLA and Sionna simulators to generate a multimodal multi-view V2X (MVX) dataset. We present novel AI-based models to predict future line-of-sight (LoS) blockages and optimal beam direction as well as an innovative antenna position optimization (APO) solution, all of which are underpinned by the multimodal dataset MVX. Our framework capitalizes on collaborative perception and significantly enhances V2X communication by integrating LiDAR and wireless data. Thorough evaluations demonstrate that our collaborative perception approach outperforms traditional methods of both beam and blockage prediction in terms of accuracy and efficiency. Additionally, we evaluate the importance of infrastructural elements in V2X systems and conduct a computational study to illustrate that our framework is suitable for various operational scenarios and can be used as a digital twin solution. This work not only contributes to the field of V2X wireless communications by providing a versatile framework for network management but also sets the stage for future research on multi-sensor fusion in AI applications for V2X wireless communication environments to enhance the efficiency and resilience of future 6G networks.