Program transformation in massively parallel systems

Abstract

The authors present two problems in mapping highly maintainable expressive parallel code manipulating multidimensional arrays in massively parallel computers: bottlenecks due to simultaneous accesses in the EREW model, and interprocessor communication. They present a source code transformation approach to solve the expressibility-high-performance problem for the multidimensional arrays designed with a four-level hierarchical design of the data types (aggregate, abstract, logical, and physical levels). A systematic method is developed to transform parallel high-level low-performance code into parallel low-level efficient ones. The method is illustrated with matrix multiplication. The method is also used to generate high-performance logical-level code for the backpropagation algorithm of neural networks that makes extensive use of matrices. The transformed code has a much higher performance than the code with a naive mapping.<>