Ctrl-C: Instruction-Aware Control Loop Based Adaptive Cache Bypassing for GPUs

Abstract

The performance of general-purpose graphics processing units (GPGPUs) is often limited by the efficiency of the memory subsystems, particularly the L1 data caches. Because of the massive multithreading computation paradigm, significant memory resource contention and cache thrashing are often observed in GPGPU workloads. This leads to high cache miss rates and substantial pipeline stall time. In order to improve the efficiency of GPU caches, we propose an instruction-aware control loop based adaptive cache bypassing design (Ctrl-C). Ctrl-C applies an instruction-aware algorithm to dynamically identify per-memory instruction cache reuse behavior. Ctrl-C then adopts feedback control loops to bypass memory requests probabilistically in order to protect cache lines with short reuse distances from early eviction. GPGPU-sim simulation based evaluation shows that Ctrl-C improves the performance of cache sensitive GPGPU workloads by 41.5%, leading to higher cache and interconnect bandwidth utilization with only an insignificant 3.5% area overhead.