Performance comparison of explainable DQN and DDPG models for cooperative lane change decision-making in multi-intelligent industrial IoT vehicles

Abstract

With the rapid advancement of intelligent connected vehicles (ICVs) technology, efficient and safe vehicular lane-changing decisions have become a focal point of interest for intelligent transportation systems (ITS). This paper investigates the application of explainable artificial intelligence (XAI) techniques to deep reinforcement learning algorithms, specifically deep Q-networks (DQN) and deep deterministic policy gradient (DDPG), for lane-changing decisions in industrial internet of things (IIoT) vehicles. By integrating innovative reward functions, the study assesses the performance differences between these models under various traffic densities and ICV counts in a three-lane highway scenario. The use of XAI feature representations enhances the transparency and interpretability of the models, providing insights into the decision-making process. XAI helps to elucidate how the models arrive at their decisions, improving trust and reliability in automated systems. The research reveals that although the DQN model demonstrates initial superior performance in the early phases of experimentation, the DDPG model outperforms in crucial performance metrics such as average fleet speed, headway, and stability during later stages of training. The DDPG model maintains better control over fleet speed and vehicle spacing in both low-density and high-density traffic environments, showcasing its superior adaptability and efficiency. These findings highlight the DDPG model's enhanced capability to manage dynamic and complex driving environments, attributed to its refined policy learning approach which adeptly balances exploration and exploitation. The novel reward function significantly promotes cooperative lane-changing behaviors among ICVs, optimizing lane change decisions and improving overall traffic flow efficiency. This study not only provides valuable technical support for lane-changing decisions in smart vehicular networks but also lays a theoretical and empirical foundation for the advancement of future ITS. The insights gained from comparing DQN and DDPG models contribute to the ongoing discussion on effective deep learning strategies for real-world ITS applications, potentially guiding future developments in autonomous driving technologies.