CorNet: Enhancing Motion Deblurring in Challenging Scenarios Using Correlation Image Sensor

Abstract

Motion deblurring in scenes involving small moving objects or low-illumination conditions is challenging. This paper presents an effective deep-learning solution that utilizes correlation images as key auxiliaries to address the problem. The correlation image, produced by a three-phase correlation image sensor (3PCIS), is a temporal correlation between incident light and reference signals within a frame time, which encodes intensity changes of incident light over the exposure time. Since correlation images explicitly record motion information lost during the blurring process during exposure, they can be used for accurately identifying the location and degree of blur, especially in low-illumination conditions and scenarios with small moving objects. Therefore, we combine correlation images and motion-blurred images as inputs and build a two-stream network for motion deblurring. Two key designs in our model are 1) Shared-gated Block (SGB), which enables information exchange between the two encoders and selectively allows useful information to pass through the network to obtain high-quality output; 2) a Motion-guided Block (MGB), decoding process that can draw more attention to the blurred areas in the image, thereby achieving clearer textures and details restoration in the blurred areas. The experimental results show that our model not only can successfully eliminate the motion blur in the above challenging scenarios, but also achieves a state-of-the-art 36.02dB in Peak Signal-to-Noise Ratio (PSNR) on the GoPro dataset with simulated correlation images.