NeuroSensor: A 3D image sensor with integrated neural accelerator

Abstract

3D integration provides opportunities to design high-bandwidth and low-power CMOS image sensors (CIS) [1–4]. The 3D stacking of pixel tier, peripheral tier, memory tier(s), and compute tier(s) enables high degree of parallel processing. Also, each tier can be designed in different technology nodes (heterogeneous integration) to further improve power-efficiency. This paper presents a case study of a smart 3D image sensor with integrated neuro-inspired computing for intelligent vision processing. Hardware acceleration of neuro-inspired computing has received much attention in recent years for recognition and classification [5]. We present the physical design of NeuroSensor, a 3D CIS with an integrated convolutional neural network (CNN) accelerator. The rationale for our approach is that 3D integration of sensor, memory, and computing will effectively harness the inherent parallelism in neural algorithms. We design the NeuroSensor considering different complexities of CNN platform, ranging from only feature extraction to complete classification, and study the trade-offs between complexity, performance, and power.