Cross-View Masked Model for Self-Supervised Graph Representation Learning

Abstract

Graph-structured data plays a foundational role in knowledge representation across various intelligent systems. Self-supervised graph representation learning (SSGRL) has emerged as a key methodology for processing such data efficiently. Recent advances in SSGRL have introduced the masked graph model (MGM), which achieves state-of-the-art performance by masking and reconstructing node features. However, the effectiveness of MGM-based methods heavily relies on the information density of the original node features. Performance deteriorates notably when dealing with sparse node features, such as one-hot and degree-hot encodings, commonly found in social and chemical graphs. To address this challenge, we propose a novel cross-view node feature reconstruction method that circumvents direct reliance on the original node features. Our approach generates four distinct views (graph view, masked view, diffusion view, and masked diffusion view) from the original graph through node masking and diffusion. These views are then encoded into representations with high information density. The reconstruction process operates across these representations, enabling self-supervised learning without direct reliance on the original features. Extensive experiments are conducted on 26 real-world graph datasets, including those with sparse and high information density environments. This cross-view reconstruction method represents a promising direction for effective SSGRL, particularly in scenarios with sparse node feature information.