Sampling-priors-augmented deep unfolding network for robust video compressive sensing

Abstract

Video Compressed Sensing (VCS) aims to reconstruct multiple frames from a single captured measurement, enabling high-speed scene recording with a low-frame-rate sensor. Despite recent advancements in VCS, state-of-the-art (SOTA) methods significantly increase model complexity and suffer from poor generality and robustness, as they require retraining to accommodate new system configurations. These limitations hinder real-time imaging and practical deployment. To address these issues, we propose a Sampling-Priors-Augmented Deep Unfolding Network (SPA-DUN) for efficient and robust VCS reconstruction. Leveraging a deep unfolding framework inspired by optimization, we introduce a lightweight and efficient U-net to reduce model size while enhancing performance. Additionally, we incorporate prior knowledge from the sampling model to dynamically modulate network features, allowing SPA-DUN to handle arbitrary sampling settings with a single model. This approach improves interpretability and generality. Extensive experiments on both simulation and real datasets demonstrate that SPA-DUN achieves SOTA performance with remarkable efficiency, offering a highly adaptable solution for VCS. Code is available at: https://github.com/yuhaoo00/SPA-DUN.