Performance Modelling of Graph Neural Networks

Abstract

Recent years have witnessed a rapid rise in the popularity of Graph Neural Networks (GNNs) that address a wide variety of domains using different architectures. However, as relevant graph datasets become diverse in size, sparsity and features, it becomes important to quantify the effect of different graph properties on the training time for different GNN architectures. This will allow us to design compute-aware GNN architectures for specific problems, and further extend this for distributed training. In this paper, we formulate the calculation of the Floating Point Operations (FLOPs) required for a single forward pass through layers of a GNN. We report the analytical calculations for GraphConv and GraphSAGE models and compare against their profiling results for 10 graphs with varying properties. We observe that there is a strong correlation between our theoretical expectation of the number of FLOPs and the experimental execution time for a forward pass.