Seg-CycleGAN: SAR-to-Optical Image Translation Guided by a Downstream Task

Abstract

Optical remote sensing and synthetic aperture radar (SAR) remote sensing are crucial for earth observation, offering complementary capabilities. While optical sensors provide high-quality images, they are limited by weather and lighting conditions. In contrast, SAR sensors can operate effectively under adverse conditions. This letter proposes a generative adversarial network (GAN)-based SAR-to-optical image translation method named Seg-CycleGAN, designed to enhance the accuracy of ship target translation by leveraging semantic information from a pretrained semantic segmentation model. Our method utilizes the downstream task of ship target semantic segmentation to guide the training of the image translation network, improving the quality of output optical-styled images. The potential of foundation-model-annotated datasets in SAR-to-optical translation tasks is revealed. This work suggests broader research and applications for downstream-task-guided frameworks. The code and link to download the proposed HRSID-DIOR dataset will be available at https://github.com/NPULHH/.