Bit-Offsetter: A Bit-serial DNN Accelerator with Weight-offset MAC for Bit-wise Sparsity Exploitation

Abstract

With the rapid evolution of deep neural networks (DNNs), the massive computational burden brings about the difficulty of deploying DNN on edge devices. This situation gives rise to specialized hardware aiming at exploiting the sparsity of DNN parameters. Bit-serial architectures (BSAs) possess great performance potential by leveraging the abundant bit-wise sparsity. However, the distribution of effective bits of weights confines the performance of BSA designs. To improve the efficiency of BSA, we propose a weight-offset multiply-accumulation (MAC) scheme and an associated hardware design called Bit-offsetter in this paper. Weight-offsetting not only significantly boosts bit-wise sparsity but also brings out a more balanced distribution of essential bits. For Bit-offsetter, aside from leveraging the abundant bitwise sparsity induced by weight-offsetting, it’s also equipped with a load-balancing scheduler to reduce idle cycles and mitigate utilization degradation. According to our experiment on a series of DNN models, weight-offsetting can increase bit-wise sparsity for pre-trained weight up to 77.4% on average. The weight-offset MAC scheme associated with Bit-offsetter achieves 3.28×/2.94× speedup/energy efficiency over the baseline.