Supporting a flexible parallel programming model on a network of workstations

Abstract

We introduce a shared memory software prototype system for executing programs with nested parallelism on a network of workstations. This programming model exhibits a very convenient and natural programming style and provides functionality similar to a subset of Compositional C++. Such programming model is especially suitable for computations whose complexity and parallelism emerges only during their execution, as in divide and conquer problems. To both support and take advantage of the flexibility inherent in the programming model, we develop an architecture, which distributes both the shared memory management and the computation, removing bottlenecks inherent in centralization, thus also providing scalability and dependability. The system supports also dynamic load balancing, and fault tolerance-both transparently to the programmer. The prototype performs well using the realistic platforms of non-dedicated network of workstation. We describe encouraging performance experiments on a network in which some of the machines became slow unpredictably (to the application program). The system coped well with such dynamic behavior.