Distributed Multi-Agent Reinforcement Learning for Cooperative Low-Carbon Control of Traffic Network Flow Using Cloud-Based Parallel Optimization

Abstract

The escalating air pollution resulting from traffic congestion has necessitated a shift in traffic control strategies towards green and low-carbon objectives. In this study, a graph convolutional network and self-attention value decomposition-based multi-agent actor-critic (GSAVD-MAC) approach is proposed to cooperative control traffic network flow, where vehicle carbon emission and traffic efficiency are considered as reward functions to minimize carbon emissions and traffic congestions. In this method, we design a local coordination mechanism based on graph convolutional network to guide the multi-agent decision-making process by extracting spatial topology and traffic flow characteristics between adjacent intersections. This enables distributed agents to make low-carbon decisions which not only account for their own interactions with the environment but also consider local cooperation with neighboring agents. Further, we design a global coordination mechanism based on self-attention value decomposition to guide multi-agent learning process by assigning various weights to distributed agents with respect to their contribution degrees. This enables distributed agents to learn a globally optimal low-carbon control strategy in a cooperative and adaptive manner. In addition, we design a cloud computing-based parallel optimization algorithm for the GSAVD-MAC model to reduce calculation time costs. Simulation experiments based on real road networks have verified the advantages of the proposed method in terms of computational efficiency and control performance.