Dual Bidirectional Feature Enhancement Network for Continuous Space-Time Video Super-Resolution

Abstract

Space-time video super-resolution aims to reconstruct the high-frame-rate and high-resolution video from the corresponding low-frame-rate and low-resolution counterpart. Currently, the task faces the challenge of efficiently extracting long-range temporal information from available frames. Meanwhile, existing methods can only produce results for a specific moment and cannot interpolate high-resolution frames for consecutive time stamps. To address these issues, we propose a multi-stage feature enhancement method that better utilizes the limited spatio-temporal information subject to the efficiency constraint. Our approach involves a pre-alignment module that extracts coarse aligned features from the adjacent odd-numbered frames in the first stage. In the second stage, we use a bidirectional recurrent module to refine the aligned features by exploiting the long-range information from all input frames while simultaneously performing video frame interpolation. The proposed video frame interpolation module concatenates temporal information with spatial features to achieve continuous interpolation, which refines the interpolated feature progressively and enhances the spatial information by utilizing the features of different scales. Extensive experiments on various benchmarks demonstrate that the proposed method outperforms state-of-the-art in both quantitative metrics and visual effects.