A 250mW 5.4G-Novel-Pixel/s Photorealistic Refocusing Processor for Full-HD Five-Camera Applications

In this paper, we present an integrated circuit which supports Full-Hd photorealistic refocusing. In contrast to the conventional single-image blurring, it provides physically-correct bokeh effect by rendering and then averaging hundreds of novel views from five images taken in different perspectives. To address the huge requirement of DRAM bandwidth and computing power, we adopt a block-based multi-rate framework and further propose two techniques: four-direction view generation and highly-parallel view rendering. The former provides a compact system architecture to save 32% of SRAM area and 92% of DRAM bandwidth without noticeable quality degradation. The latter efficiently generates 5.4G novel pixels per second to provide high-quality refocusing. This chip is fabricated in 40nm CMOS process, and the core area is 3.61 mm2. It consumes 250mW when operating at 200MHz and 0.9V to support Full-HD photorealistic refocusing up to 40 fps.