Scalable Abstractions for Parallel Programming

Abstract

Writing parallel programs that scale-that is, that naturally and efficiently adapt to the size of the problem and the number of processors available-is difficult for two reasons. First, the overhead of multiplexing the processing of data points assigned to a given processor is often great. Second, to achieve scaling in asymptotic performance, the algorithm that uses the interprocessor communication structure may need to differ from the algorithm used to process points located within an individual processor. We present abstractions intended to overcome these problems, making it straightforward to define scalable parallel program. The central abstraction is an ensemble,. which gives programmers a global view of physically distributed data, computation, and communication. We demonstrate the application of these ensembles to two variants of Batcher’s sort, describing how the concepts apply to other parallel programs.