Modelling the Impact of Cloud Storage Heterogeneity on HPC Application Performance

Abstract

Moving high-performance computing (HPC) applications from HPC clusters to cloud computing clusters, also known as the HPC cloud, has recently been proposed by the HPC research community. Migrating these applications from the former environment to the latter can have an important impact on their performance, due to the different technologies used and the suboptimal use and configuration of cloud resources such as heterogeneous storage. Probabilistic models can be applied to predict the performance of these applications and to optimise them for the new system. Modelling the performance in the HPC cloud of applications that use heterogeneous storage is a difficult task, due to the variations in performance. This paper presents a novel model based on Extreme Value Theory (EVT) for the analysis, characterisation and prediction of the performance of HPC applications that use heterogeneous storage technologies in the cloud and high-performance distributed parallel file systems. Unlike standard approaches, our model focuses on extreme values, capturing the true variability and potential bottlenecks in storage performance. Our model is validated using return level analysis to study the performance of representative scientific benchmarks running on heterogeneous cloud storage at a large scale and gives prediction errors of less than 7%.