HTMPC: A heavily templated C++ library for large scale particle-based mesoscale hydrodynamics simulations using multiparticle collision dynamics

We present HTMPC, a Heavily Templated C++ library for large-scale simulations implementing multi-particle collision dynamics (MPC), a particle-based mesoscale hydrodynamic simulation method. The implementation is plugin-based, and designed for distributed computing over an arbitrary number of MPI ranks. By abstracting the hardware-dependent parts of the implementation, we provide an identical application-code base for various architectures, currently supporting CPUs and CUDA-capable GPUs. We have examined the code for a system of more than a trillion MPC particles distributed over a few thousand MPI ranks (GPUs), demonstrating the scalability of the implementation and its applicability to large-scale hydrodynamic simulations. As showcases, we examine passive and active suspension of colloids, which confirms the extensibility and versatility of our plugin-based implementation.

Program summary

Program Title: HTMPC

CPC Library link to program files: https://doi.org/10.17632/xnxh68zhbt.1

Licensing provisions: MIT

Programming language: C++17, CUDA C++ (optional), MPI (optional)

Supplementary material: Supplementary Information, User Manual

Nature of problem: Complex fluids in soft, active, and living matter are characterized by a wide range of relevant length- and time-scales, from nanometers to millimeters, and from sub-microseconds to seconds. Their dynamics is often governed by the hydrodynamics of the embedding aqueous medium. Thus, it is essential for the numerical study of such systems to develop efficient simulation techniques and highly parallel computer codes, especially when large system sizes and emergent collective behavior are considered. Several mesoscale simulation techniques have been developed in the last decades for this purpose. Multi-particle collision dynamics (MPC), a particle-based hydrodynamics simulation technique, is a promising ansatz for such an endeavor. It is also important to develop an easy-to-extend implementation, so that the code can be adapted to various soft and living matter systems as desired.

Solution method: We develop an implementation of MPC that can exploit large-scale high-performance computing resources for hydrodynamic simulations of complex fluids. The code provides a C++ template library, which is plugin-based and can be extended by user-written plugins, implementing particles or objects interacting with the surrounding fluid. Calculations can be distributed over an arbitrary number of MPI ranks and accelerated with the current implementation supporting CUDA-capable GPUs. The code includes essential features of state-of-the-art MPC algorithms, e.g., thermostat, local angular-momentum conservation, and a variety of boundary conditions, such as periodic, no-slip (both also supporting shear flow) and slip. Simulation data can be written to and read from disk.

Additional comments including restrictions and unusual features: The code provides an option to produce perfectly reproducible particle trajectories, independent of the MPI setup of a simulation system, as long as the underlying architecture and selected features are identical. It shows good scaling behaviors for sufficiently heavy workloads even for very large problems utilizing thousands of GPUs.