Remote sensing image pan-sharpening via Pixel difference enhance

Abstract

Nowadays, embedding-based pan-sharpening networks aimed at fusing panchromatic (PAN) and multispectral (MS) images are abundant, yet their results still show spectral distortion and spatial fuzziness. In this paper, we design a multi-scale fusion structure to minimize the gap between the pan-sharpened image and the reference image progressively. Specifically, we proposed a method based on the scale difference between PAN and MS images, using a convolutional neural network embedding pixel difference enhanced module (PDEM) to obtain the pan-sharpened image and minimizing the losses from each scale. The network includes three scales, each scale contains the PDEM to generate the intermediate results until to the last scale which obtains the final pan-sharpened result. The designed PDEM extracts deep features from PAN and MS images, using different kernel sizes and receptive field scales to diversify the extracted information. Three-direction pixel difference convolutions (PDCs) are utilized to maintain and enhance the edge details of spatial information. The loss function is to sum up the mean square error and mean absolute error between the pan-sharpened image and the reference image at three scales. Extensive experiments suggest the proposed method outperforms state-of-the-art methods from visual and quantitative perspectives, and confirm the effectiveness of PDEM in extracting and retaining image information and edge enhancement. The high-level vision task experiments also show our method has good practical value for further applications.