A 1000 fps Spiking Neural Network Tracking Algorithm for On-Chip Processing of Dynamic Vision Sensor Data

Abstract

Dynamic vision sensor (DVS), an event-based camera, has attracted significant attention due to its unique characteristics. Unlike frame-based cameras, the data format of DVS makes it difficult for traditional algorithms to process it directly. On the other hand, as a new type of brain-like neural network, the spiking neural network is specially used to process spiking data, and it is well suited for this type of event-based camera. In addition, because of the rapid development of neuromorphic hardware in recent years, it is possible to deploy SNN applications on edge-side system-on-chip. Therefore, based on the characteristics of dynamic vision sensors, this paper designs a spike encoding module and an SNN for processing sensor information. We use selective search to accomplish object tracking by classifying targets and backgrounds. The SNN can achieve 98.66% classification accuracy on our synthetic test dataset, and the tracking algorithm can achieve over 1000 fps after quantizing and compiling the network to the hardware simulator.