Trans-Diff: Heterogeneous Domain Adaptation for Remote Sensing Segmentation With Transfer Diffusion

Abstract

Domain adaptation has been demonstrated to be an important technique to reduce the expensive annotation costs for remote sensing segmentation. However, for remote sensing images (RSIs) acquired from different imaging modalities with significant differences, a model trained on one modality can hardly be utilized for images of other modalities. This leads to a greater challenge in domain adaptation, called heterogeneous domain adaptation (HDA). To address this issue, we propose a novel method called transfer diffusion (Trans-Diff), which is the first work to explore the diffusion model for HDA remote sensing segmentation. The proposed Trans-Diff constructs cross-domain unified prompts for the diffusion model. This approach enables the generation of images from different modalities with specific semantics, leading to efficient HDA segmentation. Specifically, we first propose an interrelated semantic modeling method to establish semantic interrelation between heterogeneous RSIs and annotations in a high-dimensional feature space and extract the unified features as the cross-domain prompts. Then, we construct a semantic guidance diffusion model to further improve the semantic guidance of images generated with the cross-domain prompts, which effectively facilitates the semantic transfer of RSIs from source modality to target modality. In addition, we design an adaptive sampling strategy to dynamically regulate the generated images' stylistic consistency and semantic consistency. This can effectively reduce the cross-domain discrepancies between different modalities of RSIs, ultimately significantly improving the HDA remote sensing segmentation performance. Experimental results demonstrate the superior performance of Trans-Diff over advanced methods on several heterogeneous RSI datasets.