Scalable and Adaptive Graph Neural Networks with Self-Label-Enhanced Training

Abstract

Although GNNs have achieved success in semi-supervised graph learning tasks, common GNNs suffer from expensive message passing during each epoch and the exponentially growing receptive field occupying too much memory, especially on large graphs. Neighbor sampling techniques can reduce GNNs’ memory footprints, but they encounter either redundant computation or incomplete edges. Some simplified GNNs decouple graph convolutions and feature transformations to reduce computation in training. However, only a part of them can scale to large graphs without neighbor sampling techniques, which can be concluded as decoupled GNNs. Nevertheless, they either only utilize the last convolution output or simply add multi-hop features with uniform weights, which limits their expressiveness. In this paper, we refine the pipeline of decoupled GNNs and propose Scalable and Adaptive Graph Neural Networks (SAGN), which effectively leverages multi-hop information with a scalable attention mechanism. Moreover, we generalize the input of decoupled GNNs to view another classical technique, label propagation, as a special case of decoupled GNNs and propose decoupled label trick (DecLT) to incorporate label information into decoupled GNNs. Furthermore, by incorporating self-training technique, we further propose the Self-Label-Enhanced (SLE) training framework, leveraging pseudo labels to simultaneously augment the training set and improve label propagation. Extensive experiments show that SAGN outperforms other baselines, and that DecLT and SLE can consistently and significantly improve all types of models on semi-supervised node classification tasks. Many top-ranked models on Open Graph Benchmark (OGB) leaderboard adopt our methods as the main backbone.