Bridging distribution gaps: invariant pattern discovery for dynamic graph learning

Abstract

Temporal graph networks (TGNs) have been proposed to facilitate learning on dynamic graphs which are composed of interaction events among nodes. However, existing TGNs suffer from poor generalization under distribution shifts that occur over time. It is vital to discover invariant patterns with stable predictive power across various distributions to improve the generalization ability. Invariant pattern discovery on dynamic graphs is non-trivial, as long-term history of interaction events is compressed into the memory by TGNs in an entangled way, making invariant pattern discovery difficult. Furthermore, TGNs process interaction events chronologically in batches to obtain up-to-date representations. Each batch consisting of chronologically-close events lacks diversity for identifying invariance under distribution shifts. To tackle these challenges, we propose a novel method called Smile, which stands for Structural teMporal Invariant LEarning. Specifically, we first propose the disentangled graph memory network, which selectively extracts pattern information from long-term history through the disentangled memory gating and attention network. The interaction history approximator is further introduced to provide diverse interaction distributions efficiently. Smile guarantees prediction stability under diverse temporal-dynamic distributions by regularizing invariance under cross-time distribution interventions. Experimental results on real-world datasets demonstrate that Smile outperforms baselines, yielding substantial performance improvements.