Design and Implementation of SDN-enhanced MPI Broadcast Targeting a Fat-Tree Interconnect

Abstract

To meet the rising demands on high-performance computing, the number of computing nodes composing a high- performance computing system has been continuously growing. Simultaneously, the complexity of networks linking such computing nodes, or the interconnect, has also been increasing. Taking the scale-out of computing nodes in future high-performance computing systems into consideration, it is unrealistic to build more nodes with the strategy of building a network capacity sufficient enough to accommodate maximum traffic. We have worked on SDN-enhanced MPI based on the challenging idea that network traffic should be controlled based on the time-variant requirements of applications running on the high-performance computing systems. In particular, this paper aims to accelerate MPI_Bcast execution through the use of Software Defined Net-working (SDN), targeting a high-performance computing system with a Fat-tree interconnect. The MPI_Bcast proposed in this paper has the functionality of making a delivery tree of data based on traffic information obtained from SDN switches that compose the deployed interconnect. Our evaluation observed our proposed MPI_Bcast was executed up to 8.6 times faster than our previous MPI_Bcast implementation when a 700 Mbps pseudo traffic was flowed on the Fat-tree interconnect.