Behaviorally-Aware Multi-Agent RL With Dynamic Optimization for Autonomous Driving

Abstract

This study presents a novel Multi-Agent Reinforcement Learning (MURL) architecture for autonomous vehicle (AV) navigation in complex urban traffic environments. By integrating a Social Value Orientation (SVO) model into a model-free SARSA reinforcement learning framework, our approach effectively balances individual agents’ social preferences with safety and performance objectives. A logistic regression-based risk assessment module evaluates collision probabilities in real time by analyzing spatiotemporal dynamics such as distances and velocities. Additionally, a dynamic optimizer adapts the learning rate and exploration strategies of the SARSA algorithm to provide efficient convergence to optimal policies. Extensive simulation experiments demonstrate that the proposed method significantly enhances safety and efficiency, achieving a 55.6% reduction in collision risk and increasing average rewards per episode by 2.1 compared to traditional SARSA without SVO. Furthermore, the optimized policy reduces average episode length, indicating the framework’s effectiveness in providing robust decision-making and adaptability across various traffic scenarios. Note to Practitioners—The proposed framework in this paper is driven by the demand for comprehensive navigation systems in the rapidly evolving field of connected and autonomous vehicles (CAVs), especially within complex and unpredictable urban environments and mixed traffic scenarios. As AVs are getting more and more attention, the capacity to navigate effectively among many road users, including other AVs, pedestrians, and human-driven vehicles, is essential. Our framework builds upon the SARSA algorithm to produce an optimal policy for the AV and integrates a dynamic optimization method that represents the concept of risk as the inverse logistic of potential collisions. Distinctive to our proposed model is a finely-tuned Social Value Orientation (SVO) that captures the nuanced social dynamics between multiple autonomous agents, spanning a continuum from self-interested to entirely cooperative behaviors. This allows AVs to make decisions socially and cooperatively. This framework significantly influences the AV navigation sector by contributing to the development of secure, human-centric, and reliable transportation systems. Its multi-agent focus and the incorporation of dynamic optimization emphasize its potential to facilitate a network of AVs that interact with diverse road users, thus improving scalability. The robustness and adaptability of this machine learning-powered solution are crucial for navigating the varied scenarios that characterize urban driving, ensuring that AVs can adapt to changing conditions and make decisions that benefit all road users.