Image fusion for multimodality image via domain transfer and nonrigid transformation

Abstract

Infrared images can distinguish targets from their backgrounds on the basis of difference in thermal radiation, which works well at all day/night time and under all weather conditions. By contrast, visible images can provide texture details with high spatial resolution and definition in a manner consistent with the human visual system. We addressed the multimodality image fusion problem through three steps. Firstly, Domain transfer technique is introduced to transfer an image from one domain to another. For example, from visible image to infrared image. It can capture content characteristics of one image collection and figure out how these characteristics could be translated into the other image collection, all in the absence of any paired training examples. Secondly, we employ the nonrigid transformation method to match the domain transferred image and the target image, let the images pairs align in pixel level. Then we focus on fusion the domain transferred and spatial transformed image with the target image. Through translation and transformation, we simplify the fusion problem into a simple combination.