Spatio-Temporal Adaptive Weighted Fusion Network for Compressed Video Quality Enhancement

Abstract

In recent years, many deep learning-based methods for improving the quality of compressed video have emerged, some of which utilize multiple reference frames to enhance the target frame. However, most of these methods directly aggregate the temporal information of the reference frames, ignoring the spatial information within the target frame. In this brief, we propose a spatio-temporal information adaptive weighted fusion network (STAWFN) to enhance compressed video quality by dynamically integrating spatial information and temporal information. Specifically, we utilize well-designed temporal feature extractor (TFE) and spatial feature extractor (SFE) to extract temporal and spatial information, respectively. And then an adaptive weighted feature fusion module is employed to effectively fuse temporal information and spatial information. In addition, we construct multi-channel enhanced residual block to refine the fused features for better enhancement capability. Comprehensive test results on HEVC-compressed videos show that the proposed method can significantly enhance the objective and subjective quality of compressed videos and reach state-of-the-art performance.