23.9 An 8-channel 4.5Gb 180GB/s 18ns-row-latency RAM for the last level cache

Abstract

In recent years, the demand for memory performance has grown rapidly due to the increasing number of cores on a single CPU, along with the integration of graphics processing units and other accelerators. Caching has been a very effective way to relieve bandwidth demand and to reduce average memory latency. As shown by the cache feature table in Fig. 23.9.1, there is a big latency gap between SRAM caches in the CPU and the external DRAM main memory. As a key element for future computing systems, the last level cache (LLC) should have a high random access bandwidth, a low random access latency, a density of 1 to 8Gb, and all signal pads located on one side of the chip [1]. A logic-process-based solution was proposed [2], but it is not scalable, and has a high standby current due to its need for frequent refresh. HBM2 was also proposed [3], but its row latency is not better than conventional DRAM, and its random-access bandwidth is still limited by tFAW, as shown in Fig. 23.9.1. This paper describes the high-bandwidth low-latency (HBLL) RAM design: how it overcomes these challenges and meets requirements in a cost-effective way.