Safe Process Quitting for GPU Multi-Process Service (MPS)

GPUs have been widely adopted to speedup various throughput-originated applications running on HPC platforms, where typically there are a number of tasks sharing GPUs to maximize GPU utilization. To facilitate GPU sharing, GPU vendors provide tools, allowing multiple processes concurrently to use GPUs. For example, Nvidia provides MPS (Multi-Process Service) managing all GPU processes to achieve high throughput by fully exploiting hardware resources. However, such tool leads to undesired single point of failure for all GPU processes, namely, one process’s exception makes other processes abnormal. In this work, we investigate the seriousness of this GPU process interferences caused by MPS, and propose an approach to address one of these interferences, which takes place during process quitting. By using signal handling and thread synchronization techniques in this approach, GPU processes are able to quit safely without interfering other GPU processes.