A Multihierarchy Flow Field Prediction Network for Multimodal Remote Sensing Image Registration

Abstract

Multimodal remote sensing image registration aims to achieve alignment between different modal image pairs. This effectively enhances the subsequent effects of multisource data fusion, object detection and recognition, and provides support for geographic spatial analysis and applications. Most existing approaches for multimodal remote sensing image registration are targeted at registering rigid transformations accompanied by large-scale deformations. Regrettably, they overlook the local disparities between different modalities and are incapable of effectively handling scenes with nonrigid distortions. Therefore, this article proposes a multimodal remote sensing image registration method that uses multihierarchy flow field cumulative prediction at different scales. The method consists of a multiscale feature pyramid, a dense feature matching module, a swin-transformer flow field prediction, and a spatial transformation module. The model makes full use of the features of different scales and levels of the image, gradually refines the flow field prediction to align the local nonrigid distortion area, and adopts a registration strategy that combines bidirectional similarity loss and hierarchy feature registration loss for different levels of features of different modalities. At the same time, the photometric error loss is introduced to optimize the entire network from both the feature and original image levels. Experimental results show that our network model shows good registration performance for a variety of cross-modal remote sensing images with nonrigid distortion.