Traffic signal optimization control method based on attention mechanism updated weights double deep Q network

Abstract

As a critical guidance facility for vehicle convergence and diversion in urban traffic networks, the control effect of traffic signals directly affects traffic efficiency and road congestion level. As a mature deep reinforcement learning algorithm, the double deep Q network has shown a significant optimization effect in intelligent traffic signal control research. In this paper, for the feature extraction defects of deep double Q network and the problem of underestimating the evaluation value of actions, we propose an Attention Mechanism Updated Weights Double Deep Q Network (AMUW–DDQN) based on the attention mechanism for the optimal control of traffic signals. The AMUW–DDQN method enhances the perceptual ability of the network by introducing the attention mechanism of Squeeze And Excitation Networks (SENet) to make the neural network pay attention to important state components automatically, and based on the idea that accurate representation of potentially optimal action values is better than the balanced representation of all the action values, it is considered that underestimated actions have a certain probability of being the optimal action and the loss function is weighted to optimize the action values. Simulation experiments were also conducted using the traffic flow data of the intersection of Fengze Street–Tian’an South Road, Fengze District, Quanzhou City, Fujian Province, China. The experimental results show that the method proposed in this paper has the most significant final convergence effect for the same number of iterations, and has better performance in the evaluation indexes such as vehicle queue length and vehicle delay time.