Endoir: A GAN-based method for fiber bundle endoscope image restoration

Abstract

Endoscope plays a crucial role in advancing minimally invasive surgeries. Ultra-compact, agile fiber endoscopes have gained significant popularity as an alternative to traditional bulk imaging systems. They have multiple advantages, such as large field of view, long depth of field and short rigid tip length. However, these systems exhibit honeycomb-like fixed patterns (HFP) and color bias in the output images, which can be attributed to the spacing and cladding around each fiber as well as the physical structure and low light conditions. To address these issues, we propose a fiber endoscope image restoration method based on generative adversarial network (GAN) named Endoir. The generator of Endoir employs a U-Net architecture that incorporates multi-scale skip connections between the encoder and decoder. It can incorporate low-level details with high-level semantics from feature maps in different scales and reduce the number of network parameters to improve the computation efficiency. We generate a synthetic dataset by simulating the fiber endoscope image scheme using an ordinary image dataset as a basis. This approach allows us to obtain a sufficient number of image pairs with more realistic usage scenarios. Our solution not only outperforms previous methods in terms of effectively removing the HFP but also provides the capability to correct color bias. The experiment results show that our method achieves superior accuracy in removing HFP and correcting color bias compared to existing approaches.