Model-driven determination of object pose for a visually servoed robot

The National Bureau of Standards robot sensory system employs multiple hierarchical levels of sensory interpretation that interact with matching levels of world modeling. At each level, the world-modeling processes generate hypotheses about the sensory data based on a priori knowledge, prior sensory input, and knowledge of robot motion. The sensory-interpretative processes use these hypotheses to facilitate their analyses of new data. The results of the analyses are used by the world-modeling processes to correct their models of the environment. This interaction requires the development of real-time algorithms for the analysis of sensory data that can usefully employ guidance from models. This paper presents an algorithm for accomplishing this at the level of object location and pose determination. Its desirable features include the ability to deal with underconstrained problems, the ability to employ all the data in a structured-light image, and robustness in the face of several types of error and noise.