Online estimation of image Jacobian matrix by Kalman-Bucy filter for uncalibrated stereo vision feedback

Abstract

This paper studies the visual servoing problem with sensory feedback from uncalibrated stereo cameras. The linear image Jacobian matrix is used to describe the spatial and temporary approximation of the differential movement relation between image space and robotic workspace. We suggest to construct an instrumental dynamic system with state variables formed from elements of the image Jacobian matrix. Thus a Kalman-Bucy filter is used to estimate the state variables of the constructed system online, which is robust to system noise and external disturbances. A 3D tracking task by a robot manipulator with visual feedback from uncalibrated stereo cameras is exemplified to show the formation of the instrumental system, estimation process and performance of the image Jacobian matrix and the design of the servo controller. Effectiveness of the proposed method and satisfactory tracking process can be found from extensive simulations and experiments provided in the paper.