SPLP: A Certifiably Globally Optimal Solution to the Relative Pose Estimation Problem Using Points and Line Pairs

Estimating the relative pose between two calibrated views with 2D-to-2D correspondences is a fundamental problem in computer vision and 2D perception. In this paper, we present the first certifiably globally optimal solver that can simultaneously incorporate both points and lines as the non-minimal 2D-to-2D correspondences for this problem. Our first contribution is to derive a generalized polynomial-based objective function based on the geometric constraints of orthogonal and parallel line pairs. Built upon it, our second contribution is to reformulate the relative pose estimation problem as a constrained global optimization problem with a unified representation of both point and line pair correspondences. Our third contribution lies in relaxing this non-convex optimization problem to a convex Semi-Definite Program (SDP) using Sum of Squares (SOS) relaxations so as to solve it via Gloptipoly 3 with a reliable guarantee of global optimality. In both synthetic and real experiments, we show that adopting line pairs as supplementary correspondences can greatly improve estimation accuracy, especially in the point-sparse situations, and that our solver, named SPLP (SOS-Point-and-Line-Pair), can outperform other state-of-the-art solvers.