CGR-Net: Consistency Guided ResFormer for Two-View Correspondence Learning

Abstract

Accurately identifying correct correspondences (inliers) in two-view images is a fundamental task in computer vision. Recent studies usually adopt Graph Neural Networks or stack local graphs into global ones to establish neighborhood relations. However, the smoothing properties of Graph Convolutional Neural network (GCN) cause the model to fall into local extreme, which leads to the issue of indistinguishability between inliers and outliers. Especially when the initial correspondences contain a large number of incorrect correspondences (outliers), these studies suffer from severe performance degradation. To address the above issues and refocus perspective information on distinct features, we design a Consistency Guided ResFormer Network (CGR-Net) that uses consistent correspondences to guide model perspective focusing, thereby avoiding the negative impact of outliers. Specifically, we design an efficient Graph Score Calculation module, which aims to compute global graph scores by enhancing the representation of important features and comprehensively capturing the contextual relationships between correspondences. Then, we propose a Consistency Guided Correspondences Selection module to dynamically fuse global graph scores and consistency graphs and construct a novel consistency matrix to accurately recognize inliers. Extensive experiments on various challenging tasks demonstrate that our CGR-Net outperforms state-of-the-art methods. Our code is released at https://github.com/XiaojieLi11/CGR-Net .