An Energy-Efficient POSIT Compute-in-Memory Macro for High-Accuracy AI Applications

Abstract

Floating-point (FP) CIM is an attractive technique that achieves high accuracy with considerable energy efficiency, especially for complex artificial intelligence tasks. However, FP-CIM with FP32/FP16/BF16 data format incurs a performance bottleneck due to the large storage requirements and MAC power. The emerging POSIT data format exploits dynamic bit width that adapts to varied data distributions, enabling using a low-bit-width to reach nearly the same accuracy performance as conventional high-bit-width FP (POSIT $8~{\approx }$ FP16/BF16). Therefore, a POSIT-based CIM exhibits inherent advantages over the traditional FP-CIM. Despite the theoretical superiority, the dynamic property of POSIT introduces huge exponent processing overhead, cell underutilization of the CIM array, and redundant logic toggles in mantissa addition for POSIT-base CIM. This article presents a POSIT-based digital CIM (PD-CIM) macro with three features to tackle the above challenges: 1) a bi-directional regime processing unit (BRPU) simplifies the complicated codec logic to reduce the energy overhead in dynamic exponent processing; 2) a critical-bit pre-compute-and-store (CPCS) CIM utilizes the spare CIM cells caused by the dynamic bit-width of mantissa to improve cell utilization of the CIM array; and 3) a cyclically alternating computing-scheduling unit replaces the bit-wise addition of dynamically aligned mantissas with bit-wise OR to save logic toggles power. Fabricated in a 28 nm CMOS technology, PD-CIM occupies an area of 1.41 mm2. It reaches an 83.23 TFLOPS/W peak energy efficiency at POSIT(8,1) for ImageNet classification on the vision transformer base (ViT-B) model. Compared with the state-of-the-art FP-CIM, PD-CIM reduces energy by $2.36{\times }$ and offers $3.51{\times }$ speedup.