Maximal clique enumeration for large graphs on hadoop framework

Abstract

Maximal clique enumeration (MCE) problem for very large graphs appears in many critical applications such as community detection in social networks, aligning 3D protein sequences, finding motifs in genomic data, identifying co-expressed genes and data analytics in communication networks. It is not unusual to have graphs of billions of nodes and edges in these applications. The MCE problem is NP hard, but a number of algorithms both sequential and parallel have been proposed that work efficiently for real graphs. In addition to the large sizes of the input graphs, the MCE algorithms in general result in large intermediate data making it even more challenging to efficiently process the data. Recently an approach has been proposed, referred to as pbitMCE, which is shown to outperform or perform equally well compared to the existing approaches. The approach uses degeneracy ordering of vertices which plays a vital role in the performance of the algorithm. Degeneracy ordering of vertices can be generated in linear time. However it is challenging to find the degeneracy ordering in a distributed environment as it requires extensive communication between the nodes. In some cases generating the ordering can take a significant amount of time. In such cases a different ordering such as ordering by degree can be a better choice than the degeneracy ordering. In this paper we experimentally study the impact of various ordering of vertices on the performance of an MCE algorithm in the context of mapreduce framework. We present an implementation of pbitMCE using mapreduce that takes a large graph and an ordering of vertices as input and enumerates all the maximal cliques. To support the study, we present the experimental results on various graphs using different orderings. The results show that the degree ordering performs comparable to the degeneracy ordering in most cases while it performs poorer in the case of large graphs.