SCMA: A Sparse Cooperative Multi-Agent Framework for Adaptive Traffic Signal Control

Building scalable, adaptive, and collaborative traffic signal control system still remains to be further explored across relevant research communities, including computer science and transportation groups. In this study, a scalable multi-agent framework is proposed based on the coordination graphs framework where the global objective is decomposed into a linear sum of local edge-based functions. The proposed edge-based decomposition scales linearly with edges in dense networks. A novel combination of max-plus joint action selection algorithm with two collaborative model-free methods, including sparse cooperative Q-learning (SparseQ) and relative sparse cooperative Q-learning (RSparseQ), is utilized to control multi-intersection networks. Extensive experiments are carried out, and their results demonstrate the effectiveness of our proposed framework. In comparison with independent Q-learning agents, our proposed framework achieves superior performance in terms of vehicle trip time, waiting time and jam length. In addition, the reported results show that the proposed RSparseQ outperforms SparseQ in avoiding vehicles teleports, which leads to better driver satisfaction.