Disparity-Guided Multi-View Interaction Network for Light Field Reflection Removal

Abstract

Light field (LF) imaging presents a promising avenue for reflection removal, owing to its ability of reliable depth perception and utilization of complementary texture details from multiple sub-aperture images (SAIs). However, the domain shifts between real-world and synthetic scenes, as well as the challenge of embedding transmission information across SAIs pose the main obstacles in this task. In this paper, we conquer the above challenges from the perspectives of data and network, respectively. To mitigate domain shifts, we propose an efficient data synthesis strategy for simulating realistic reflection scenes, and build the largest ever LF reflection dataset containing 420 synthetic scenes and 70 real-world scenes. To enable the transmission information embedding across SAIs, we propose a novel D isparity-guided M ulti-view I nteraction Net work (DMINet) for LF reflection removal. DMINet mainly consists of a transmission disparity estimation (TDE) module and a center-side interaction (CSI) module. The TDE module aims to predict transmission disparity by filtering out reflection disturbances, while the CSI module is responsible for the transmission integration which adopts the central view as the bridge for the propagation conducted between different SAIs. Compared with existing reflection removal methods for LF input, DMINet achieves a distinct performance boost with merits of efficiency and robustness, especially for scenes with complex depth variations.