Addressing Non-Co-Visible Regions in SAR and Optical Image Registration: A Two-Stage Matching and Ensemble Method

In scene matching navigation, differences in trajectory, angle, time, and imaging mechanisms between optical sensors and synthetic aperture radar (SAR) can result in non-co-visible areas between SAR and optical images. This means that features may be mismatched with features in visible areas of another image, leading to incorrect feature correspondence and reduced overall matching quality, which can cause positioning errors. In addition, processing non-co-visible regions can significantly increase the computational cost of feature extraction, description, and matching, particularly for large images. While existing matching algorithms are less sensitive to non-co-visible regions between SAR and optical images, they may not be effective enough to accurately match images in all scenarios. To address this problem, this article proposes a two-stage matching method. First, pretrained deep matching models are used to extract matching key point pairs from SAR–optical images, and then a clustering algorithm groups matching key points in SAR images and optical images into clusters, extracting the top 70%–90% of matching keypoints. Bounding boxes (co-visible area) are generated for each image based on the extracted keypoints. In the second stage, a multiscale, multimodel ensemble approach is employed to extract matching keypoint pairs for co-visible regions. Finally, the matching key points obtained in the two stages are concatenated to produce a comprehensive set of corresponding features between the SAR and optical images. Extensive experiments demonstrate the efficacy of the proposed method in SAR–optical image registration.