High Throughput VLSI Architecture for Image Pyramid Generation in Computer Vision

Abstract

Image Pyramids i.e. set of down-sampled images from high-resolution input is used in computer vision processing pipe to account for unknown distance of objects from Camera. The image pyramid can be set of Octave (down-sampling by 2) and/or Generic scaling (arbitrary downscaling ratio). The prior literature addresses generating all down-scaled images using either input or previous output as an overall scaling architecture and consists of "N" set of independent scalers, which are separately tuned to Octave and generic scaling. The disadvantage of the above approach is higher silicon area and DRAM Bandwidth proportional to number of independent scalars. The paper proposes a novel solution on top of traditional poly-phase filtering which consists of new concepts e.g. Re-scale from previous octave scale architecture, Multi-thread processing with flexible mapping of shared Scalers, Unconventional processing order for 2D scaling without line buffers, shared coefficients and flexible Region of Interest (ROI). The proposed solution is implemented as HW IP with 0.2 mm2 in 16nm process node with 720 Mpix/sec throughput, which is 3.5X lower in the area and 40% lower DRAM bandwidth compared to prior literature.