RGVPSeg: multimodal information fusion network for retinogeniculate visual pathway segmentation.

Abstract

The segmentation of the retinogeniculate visual pathway (RGVP) enables quantitative analysis of its anatomical structure. Multimodal learning has exhibited considerable potential in segmenting the RGVP based on structural MRI (sMRI) and diffusion MRI (dMRI). However, the intricate nature of the skull base environment and the slender morphology of the RGVP pose challenges for existing methodologies to adequately leverage the complementary information from each modality. In this study, we propose a multimodal information fusion network designed to optimize and select the complementary information across multiple modalities: the T1-weighted (T1w) images, the fractional anisotropy (FA) images, and the fiber orientation distribution function (fODF) peaks, and the modalities can supervise each other during the process. Specifically, we add a supervised master-assistant cross-modal learning framework between the encoder layers of different modalities so that the characteristics of different modalities can be more fully utilized to achieve a more accurate segmentation result. We apply RGVPSeg to an MRI dataset with 102 subjects from the Human Connectome Project (HCP) and 10 subjects from Multi-shell Diffusion MRI (MDM), the experimental results show promising results, which demonstrate that the proposed framework is feasible and outperforms the methods mentioned in this paper. Our code is freely available at https://github.com/yanglin9911/RGVPSeg .