An Efficient Brain-Inspired Accelerator Using a High-Accuracy Conversion Algorithm for Spiking Deformable CNN

Abstract

Spiking Neural Network (SNN), inspired by the brain, has shown promising potential in terms of low-power deployment on resource-constrained devices. The SNN can be obtained by two approaches: training from scratch or conversion from existing Artificial Neural Network (ANN). However, the directly training SNN often leads to suboptimal accuracy. Therefore, methods based on converting existing ANN have become the preferred choice for achieving high accuracy. To enhance the feature-capturing capability of the converted SNNs, various operations, such as transposed convolution and deformable convolution, have been introduced, which bring multiple challenges to conversion algorithms and hardware designs. In this brief, we propose a universal SNN conversion method for deformable convolution to enhance the modeling capability of receptive fields for spatial information. The proposed conversion algorithm not only maintains high accuracy but also makes converted deformable convolutions highly hardware-efficient. Building upon the deformable SNN, we develop a low-complexity processing element and computing array, enabling flexible execution of complex and heterogeneous operations within deformable SNNs without requiring any multipliers. In addition, the overall architecture with energy-efficient dataflow is designed for our deformable SNN model and is implemented in TSMC 28-nm HPC+ technology node. Experiments show that the proposed conversion algorithm suffers negligible accuracy degradation in the challenging object detection task. The accelerator achieves at least $1.2\times $ higher energy efficiency compared to previous designs while maintaining 47.9% mAP.