TIG-UDA: Generative unsupervised domain adaptation with transformer-embedded invariance for cross-modality medical image segmentation

Abstract

Unsupervised domain adaptation (UDA) in medical image segmentation aims to transfer knowledge from a labeled source domain to an unlabeled target domain, especially when there are significant differences in data distribution across multi-modal medical images. Traditional UDA methods typically involve image translation and segmentation modules. However, during image translation, the anatomical structure of the generated images may vary, resulting in a mismatch of source domain labels and impacting subsequent segmentation. In addition, during image segmentation, although the Transformer architecture is used in UDA tasks due to its superior global context capture ability, it may not effectively facilitate knowledge transfer in UDA tasks due to lacking the adaptability of the self-attention mechanism in Transformers. To address these issues, we propose a generative UDA network with invariance mining, named TIG-UDA, for cross-modality multi-organ medical image segmentation, which includes an image style translation network (ISTN) and an invariance adaptation segmentation network (IASN). In ISTN, we not only introduce a structure preservation mechanism to guide image generation to achieve anatomical structure consistency, but also align the latent semantic features of source and target domain images to enhance the quality of the generated images. In IASN, we propose an invariance adaptation module that can extract the invariability weights of learned features in the attention mechanism of Transformer to compensate for the differences between source and target domains. Experimental results on two public cross-modality datasets (MS-CMR dataset and Abdomen dataset) show the promising segmentation performance of TIG-UDA compared with other state-of-the-art UDA methods.