WtNGAN: Unpaired image translation from white light images to narrow-band images

Abstract

As one of the most dangerous cancers, gastric cancer poses a serious threat to human health. Currently, gastroscopy remains the preferred method for gastric cancer diagnosis. In gastroscopy, white light and narrow-band light image are two necessary modalities providing deep learning-based multimodal-assisted diagnosis possibilities. However, there is no paired dataset of white-light images (WLIs) and narrow-band images (NBIs), which hinders the development of these methods. To address this problem, we propose an unpaired image-to-image translation network for translating WLI to NBI. Specifically, we first design a generative adversarial network based on Vision Mamba. The generator enhances the detailed representation capability by establishing long-range dependencies and generating images similar to authentic images. Then, we propose a structural consistency constraint to preserve the original tissue structure of the generated images. We also utilize contrastive learning (CL) to maximize the information interaction between the source and target domains. We conduct extensive experiments on a private gastroscopy dataset for translation between WLIs and NBIs. To verify the effectiveness of the proposed method, we also perform the translation between T1 and T2 magnetic resonance images (MRIs) on the BraTS 2021 dataset. The experimental results demonstrate that the proposed method outperforms state-of-the-art methods.