Smoothing on Dynamic Concurrency Throttling

Abstract

Technology scaling has been allowing a growing number of cores in processors to satisfy the increasing demand of new applications, which need to process huge amounts of data in High-Performance Computing (HPC). However, considering that many parallel applications have limited scalability, not always activating the maximum number of available cores to execute an application will provide the best outcome in energy and performance (represented by the Energy-Delay Product, or EDP). Because of that, many works have already proposed different Dynamic Concurrency Throttling (DCT) techniques to adapt thread count as the application executes. However, dynamically tuning (i.e. increasing or decreasing) thread count implies in overheads related to caches warm-up and thread reallocation across the cores. This overhead may become very significant when thread count changes often during execution, which may overcome the benefits brought by DCT. This problem is further aggravated in Non-Uniform Memory Access (NUMA) systems since some cores are more distant, in terms of latency, than others. With that in mind, in this paper, we propose a smoothing-based strategy to minimize the thread count changes and, consequently, mitigate the aforementioned overhead. Our proposal is generic and aims further to improve the optimization results of any DCT technique. As case-study, we performed experiments on two multicore systems with nine well-known benchmarks, showing that our smoothing technique improves EDP results of offline and online state-of-the-art DCT techniques by up to 93% and 89% (both 22% on mean), respectively.