Event-based video reconstruction via attention-based recurrent network

Abstract

Event cameras are novel sensors that capture brightness changes in the form of asynchronous events rather than intensity frames, offering unique advantages such as high dynamic range, high temporal resolution, and no motion blur. However, the sparse, asynchronous nature of event data poses significant challenges for visual perception, limiting compatibility with conventional computer vision algorithms that rely on dense, continuous frames. Event-based video reconstruction has emerged as a promising solution, though existing methods still face challenges in capturing fine-grained details and enhancing contrast. This paper presents a novel approach to video reconstruction from asynchronous event streams, leveraging the unique properties of event data to produce high-quality video. Our method integrates channel and pixel attention mechanisms to focus on essential features and incorporates deformable convolutions and adaptive mix-up operations to provide flexible receptive fields and dynamic fusion across down-sampling and up-sampling layers. Experimental results on multiple real-world event datasets demonstrate that our approach outperforms comparable methods trained on the same dataset, achieving superior video quality from pure event data. We also demonstrate the capability of our method for high dynamic range reconstruction and color video reconstruction using an event camera equipped with a Bayer filter.