An 800-MHz 8.17-TOPS/W 0.63-TOPS/mm2 Memory-Utilization-Aware CNN Accelerator Featuring a Memory Stationary Dataflow

Abstract

Increasing the on-chip memory utilization (OCMU) is crucial for an area-efficient deep neural network accelerator. We propose a memory stationary (MS) dataflow to ingeniously combine the input and output features in a single memory block in a cyclic manner, significantly increasing the OCMU. The MS dataflow also reduces the feature memory access by 78.0%. Furthermore, residual paths in a ResNet model require large feature buffering. We introduce layer-wise clipped-asymmetric residual distillation (LCARD) quantization, removing the residual paths with minimal accuracy degradation. It dynamically assigns different feature/weight bit-widths for different layers, further enhancing the OCMU by $3.2{\times }$ and throughput by $4.5{\times }$ from a fixed bit-width (FBW) approach. We also present an MS gating (MSG) to skip the ineffective channels that improve the OCMU by $1.2{\times }$ and throughput by $1.3{\times }$ . Fabricated in a 28-nm CMOS process, the proposed accelerator exhibits an 8.17-TOPS/W peak energy efficiency and a 0.63-TOPS/mm2 peak area efficiency at 800 MHz and 0.9 V while requiring only a 120 kB on-chip static random-access memory (SRAM) for the ResNet-50.