Balancing interprocessor communication and computation on torus-connected multicomputers running compiler-parallelized code

The machine model considered in this paper is that of a distributed memory parallel processor (DMPP) with a two-dimensional torus topology. Within this framework, the authors study the relationship between the speedup delivered by compiler-parallelized code and the machine's interprocessor communication speed. It is shown that compiler-parallelized code often exhibits more interprocessor communication than manually parallelized code and that the performance of the former is therefore more sensitive to the machine's interprocessor communication speed. Because of this, a parallelizing compiler developed for a platform not explicitly designed to sustain the increased interprocessor communication will produce-in the general case-code that delivers disappointing speedups. Finally, the study provides the point of diminishing return for the interprocessor communication speed beyond which the DMPP designer should focus on improving other architectural parameters, such as the local memory-processor bandwidth.<>