GNN-transformer contrastive learning explores homophily

Abstract

Graph Contrastive Learning (GCL) leverages graph structure and node feature information to learn powerful node representations in a self-supervised manner, attracting significant attention from researchers. Most GCL frameworks typically use Graph Neural Networks (GNNs) as their foundational encoders. Still, GNN methods have inherent drawbacks: local GNNs struggle to capture long-range dependencies, and deep GNNs face the oversmoothing problem. Moreover, existing GCL methods do not adequately model node feature information, relying on topology to learn neighbor features. In this paper, we introduce a novel contrastive learning mechanism that employs transformers to capture long-range dependency information while integrating the strong perceptual capabilities of GNNs for local topology, resulting in a GCL architecture that is highly robust across different levels of homophily. Specifically, we design three views: the original view, the long-range information view, and the feature view. By jointly contrasting these three views, the model effectively acquires rich information from the graph. Experimental results on seven real-world datasets with varying levels of homophily demonstrate that the proposed method significantly outperforms other baseline models, validating its effectiveness and rationality.