A 250-mW 5.4G-Rendered-Pixel/s Realistic Refocusing Processor for High-Performance Five-Camera Mobile Devices

Abstract

Digital refocusing in multicamera mobile devices is becoming crucial. Realistic refocusing, which is a subset of digital refocusing, provides physically correct quality; however, its intense computational complexity results in low processing speed and restricts its applicability. Moreover, its complex computation flow requires substantial DRAM bandwidth and a large SRAM area, making it more challenging to implement in hardware. In this article, we present a high-performance refocusing processor based on a hardware-oriented realistic refocusing algorithm. The proposed compact computation flow saves 92% of the DRAM bandwidth and 32% of the SRAM area without noticeable quality degradation. To support high-performance refocusing, we develop highly paralleled engines for view rendering. They deliver 5.4G rendered-pixel/s throughput. The hardware accelerator improves the processing speed by $100\times $ to $350\times $ that of the original refocusing algorithm running on a general-purpose processor. The chip is fabricated with 40-nm CMOS technology and comprises 271 kB of SRAM and 2.3M logic gates. The chip processes Full-HD light fields up to 40 frames/s under 250 mW power consumption.