PIMSR: An Energy-Efficient Processing-in-Memory Accelerator for 60 FPS 4K Super-Resolution

Abstract

Due to the huge computational load, CNN-based super-resolution solutions are hard to be deployed on resource-constrained edge devices. In this brief, we proposed a Processing-in-Memory based SR accelerator (PIMSR) that leverages direct simple memory access operations to supersede intensive multiply-add in convolution operators, thus fundamentally improving the energy efficiency. To address the storage explosion problem, we presented an index reparameterized strategy for shrinking the memory requirement of LUT-based PIMSR framework by $19.58\times $ . Furthermore, an address remapping design is putted forward to solve the access conflict in overlap configuration, which greatly boosts the efficiency of memory access. The prototype validation on low-end XC7A200T FPGA indicates that our design yields a new record of energy efficiency up to 671.61 Mpixels/J, over $5.36\times $ higher than existing deep learning based image processors, with extremely low hardware costs.