Reconfigurable Neural Synaptic Plasticity-Based Stochastic Deep Neural Network Computing

Abstract

With the increasing popularity of deep neural networks (DNNs), a large amount of research effort has been devoted to the hardware acceleration of DNNs to achieve efficient processing. Nevertheless, few works have explored the similarities between the biological essence of DNNs and arithmetic circuits. Moreover, stochastic computing (SC), which implements complex arithmetic operations with simple logic gates, has been applied to the acceleration of DNNs. However, traditional SC suffers from high latency and large hardware cost of pseudo-random number generators (PRNGs). Inspired by neural synaptic plasticity and SC, in this work, we present the reconfigurable neural synaptic plasticity-based computing (RNSP) to mimic the biological neuron behaviors and exploit the parallelism of SC to the full extent while maintaining a small hardware footprint compared to fixed-point counterparts. RNSP converts fixed-point numbers to parallel bits without logic resources, which are then synthesized by bit-wise multiplications and some full adders. In addition, we propose the arithmetic unit based on RNSP and use re-training to mitigate the accuracy degradation. Finally, a convolution engine (CE) built on RNSP with high memory bandwidth efficiency is designed. According to the implementation results on FPGA, the proposed RNSP-based CE outperforms the fixed-point counterpart in terms of power consumption and area.