Video frame interpolation based on depthwise over-parameterized recurrent residual convolution

Abstract

To effectively address the challenges of large motions, complex backgrounds and large occlusions in videos, we introduce an end-to-end method for video frame interpolation based on recurrent residual convolution and depthwise over-parameterized convolution in this paper. Specifically, we devise a U-Net architecture utilizing recurrent residual convolution to enhance the quality of interpolated frame. First, the recurrent residual U-Net feature extractor is employed to extract features from input frames, yielding the kernel for each pixel. Subsequently, an adaptive collaboration of flows is utilized to warp the input frames, which are then fed into the frame synthesis network to generate initial interpolated frames. Finally, the proposed network incorporates depthwise over-parameterized convolution to further enhance the quality of interpolated frame. Experimental results on various datasets demonstrate the superiority of our method over state-of-the-art techniques in both objective and subjective evaluations.