A generative adversarial network based on deep supervision for anatomical and functional image fusion

Medical image fusion techniques improve single-image representations by integrating salient information from medical images of different modalities. However, existing fusion methods suffer from limitations, such as vanishing gradients, blurred details, and low efficiency. To alleviate these problems, a generative adversarial network based on deep supervision (DSGAN) is proposed. First, a two-branch structure is proposed to separately extract salient information, such as texture and metabolic information, from different modal images. Self-supervised learning is performed by building a new deep supervision module to enhance effective feature extraction. The fusion and multimodal input images are then placed in the discriminator for computation. Adversarial loss based on the Earth Mover’s distance ensures that more spatial frequency, gradient, and contrast information are maintained in a fusion image, and makes model training more stable. In addition, DSGAN is an end-to-end model that does not manually set up complex fusion rules. Compared with classic fusion methods, the proposed DSGAN retains rich texture details and edge information in the input image, fuses images faster, and exhibits superior performance in objective evaluation metrics.