Combining rule decomposition and data partitioning in parallel datalog program processing

Abstract

There are two approaches to processing Datalog programs in parallel. One is to decompose the rules of a program into concurrent modules, and then assign them to processors. The other is to partition data between processors, so that each processor evaluates the same program, but with less data. The authors propose a third approach which combines the two methods in a single framework. In this approach, rules are decomposed into segments and data is partitioned among the segments. There are a number of advantages of this approach. Most importantly, it provides good focus on processing the tuples that are relevant to queries, and allows data to be partitioned and balanced dynamically at different levels. An analytic performance study is also presented to illustrate the usefulness of the proposed approach.<>