PMCN: Parallax-motion collaboration network for stereo video dehazing

Abstract

Despite progress in learning-based stereo dehazing, few studies have focused on stereo video dehazing (SVD). Existing methods may fall short in the SVD task by not fully leveraging multi-domain information. To address this gap, we propose a parallax-motion collaboration network (PMCN) that integrates parallax and motion information for efficient stereo video fog removal. We delicately design a parallax-motion collaboration block (PMCB) as the critical component of PMCN. Firstly, to capture binocular parallax correspondences more efficiently, we introduce a window-based parallax attention mechanism (W-PAM) in the parallax interaction module (PIM) of PMCB. By horizontally splitting the whole frame into multiple windows and extracting parallax relationships within each window, memory usage and runtime can be reduced. Meanwhile, we further conduct horizontal feature modulation to handle cross-window disparity variations. Secondly, a motion alignment module (MAM) based on deformable convolution explores the temporal correlation in the feature space for an independent view. Finally, we propose a fog-adaptive refinement module (FARM) to refine the features after interaction and alignment. FARM incorporates fog prior information and guides the network in dynamically generating processing kernels for dehazing to adapt to different fog scenarios. Quantitative and qualitative results demonstrate that the proposed PMCN outperforms state-of-the-art methods on both synthetic and real-world datasets. In addition, our PMCN also benefits the accuracy improvement for high-level vision tasks in fog scenes, e.g., object detection and stereo matching.