Unveiling node relationships for traffic forecasting: A self-supervised approach with MixGT

Abstract

In traffic forecasting, a key challenge lies in capturing both long-term temporal dependencies and inter-node relationships. While recent work has addressed long-term dependencies using Transformer-based models, the handling of inter-node relationships remains limited. Most studies rely on predefined or adaptive adjacency matrices, which fail to capture rich, dynamic relationships such as traffic similarity and strength, features embedded in time-varying data and challenging to model effectively. To comprehensively understand and leverage these inter-node relationships, we propose a unified framework: Pretrained Graph Transformer (PreGT) and Mix Graph Transformer (MixGT). PreGT, through self-supervised masking and reconstruction of nodes, learns latent representations of inter-node relationships from time-varying node features. MixGT integrates relationship matrix construction and utilization modules, effectively leveraging the latent representations from PreGT through graph convolution and attention mechanisms to enhance the model’s ability to capture dynamic inter-node relationship features. Experimental validation on real traffic flow datasets demonstrates the effectiveness of our framework in predicting traffic flow by accurately capturing inter-node relationships.