Refining 3D reconstruction: a theoretical and experimental study of the effect of cross-correlations

Abstract In robot navigation a model of the environment needs to be reconstructed for various applications, including path planning, obstacle avoidance, and determining where the robot is located. Traditionally, the model was acquired using two images (two-frame structure from motion) but the acquired models were unreliable and inaccurate. Recently, research has shifted to using several frames (multiframe structure from motion) instead of just two frames. However, almost none of the reported multiframe algorithms have produced accurate and stable reconstructions for general robot motion. The main reason seems to be that the primary source of error in the reconstruction-the error in the underlying motion-has been mostly ignored. Intuitively, if a reconstruction of the scene is made up of points, this motion error affects each reconstructed point in a systematic way. For example, if the translation of the robot is erroneous in a certain direction, all the reconstructed points would be shifted along the same direction. The contributions of this paper include mathematically isolating the effect of the motion error (as correlations in the structure error) and showing theoretically that these correlations can improve existing multiframe structure from motion techniques. Finally it is shown that new experimental results and previously reported work confirm the theoretical predictions.