Implementation and optimisation of the cdugksFoam solver on the Sunway TaihuLight supercomputer

Abstract

In this study, the cdugksFoam solver was successfully implemented on the Sunway TaihuLight system using the MPI + Athread programming model. To utilise the heterogeneous SW26010 many-core processor fully, we implemented three levels of parallelisation: MPI process-level hybrid parallelisation in physical space and velocity space, thread-level parallelisation to further partition physical space, and single-instruction multiple-data (SIMD) vectorisation. To address the performance bottleneck caused by the low memory bandwidth of the SW26010 processor, a series of optimisation methods, including kernel fusion, transcendental function optimisation, and soft cache, were designed and implemented to reduce discrete memory access and improve the computational efficiency of CPEs. The accuracy of the optimised program was validated through simulations of the 3D lid-driven cavity flow and rarefied supersonic flow past a sphere. Based on the experimental results from multiple grid scales, the overall performance achieved an acceleration of over 5 times compared to running on the management processing elements. In both strong and weak scalability tests, a parallel efficiency exceeding 90% was achieved.