ReGraph

Abstract

"Think Like a Sub-Graph (TLASG)" is a philosophy proposed for guiding the design of graph-oriented programming models. As TLASG-based models allow information to flow freely inside a partition, they usually require much fewer iterations to converge when compared with "Think Like a Vertex (TLAV)"-based models. In this paper, we further explore the idea of TLASG by enabling users to 1) proactively repartition the graph; and 2) efficiently scale down the problem's size. With these methods, our novel TLASG-based distributed graph processing system ReGraph requires even fewer iterations (typically ≤ 6) to converge, and hence achieves better performance (up to 45.4X) and scalability than existing TLAV and TLASG-based frameworks. Moreover, we show that these optimizations can be enabled without a large change in the programming model. We also implement our novel algorithm on top of Spark directly and compare it with other Spark-based implementation, which shows that our speedup is not bounded to our own platform.