Memory-based walk-enhanced dynamic graph neural network for temporal graph representation learning

Abstract

Depending on the ability of obtaining low-dimensional representations of nodes that preserve valuable structural information, graph representation learning has a wide range of applications in graph analysis and inference. However, real-world complex systems are naturally heterogeneous and time-varying, which makes it difficult to learn high-quality node representations. We propose a Memory-based Walk-enhanced Dynamic Graph neural Network (denoted as MWDGN) to fully exploit the dependencies and structural features in temporal graph. To capture long-term dependencies, we use a memory module to store and evolve dynamic node representations. MWDGN captures network structural information by constructing time-constrained walk sequences for each interaction node. The walk sequence features are creatively integrated into the update process of memory module, so as to capture the useful information of neighborhood structure features for the interaction node while preserving the long-term dependency of the temporal graph. In addition, we focus on the enlightenment of non-negligible temporal information for sensing key historical interaction nodes of the target node, and design a new aggregation method of historical interaction nodes information. It exploits the temporal attenuation effect of event impact to model short-term dependencies. We further exploit causal convolutional network to mine the potential associations of historical interaction node features of the target node. Comparison experiments on six datasets with mainstream baseline models demonstrate that MWDGN is capable of jointly extracting the heterogeneity and evolutionary patterns of nodes in the graph, improving the node representation quality, and enhancing the performance of the temporal link prediction and dynamic node classification tasks. The effectiveness of the proposed model is further proved by time complexity analysis, ablation study and parameter sensitivity analysis.