mpiPython: A Robust Python MPI Binding

Abstract

For the last two decades, Python has become one of the most popular programming languages and been used to develop and analyze data-intensive scientific and engineering applications and in the areas such as Bigdata Analytics, Social Media, Data Science, Physics, Psychology, Healthcare, Political Science, etc. Moreover, demand of supporting Python data-parallel applications for those areas is rapidly growing. There have been international efforts to produce a message passing interface for Python bindings to support parallel computing, but specific challenges still remain to improve Python bindings. The main purpose of this paper is to introduce our MPI Python binding, called mpiPython, with the MPI standard communication API. In this paper, we first will discuss the design issues of the mpiPython API, associated with its development. In the second part of the paper, we will discuss node/parallel performance to compare mpiPython to other MPI bindings on a Linux cluster and can expect mpiPython achieves quite acceptable performance.