Contact-consistent visual object pose estimation for contact-rich robotic manipulation tasks

Purpose The purpose of this paper is to estimate the contact-consistent object poses during contact-rich manipulation tasks based only on visual sensors. Design/methodology/approach The method follows a four-step procedure. Initially, the raw object poses are retrieved using the available object pose estimation method and filtered using Kalman filter with nominal model; second, a group of particles are randomly generated for each pose and evaluated the corresponding object contact state using the contact simulation software. A probability guided particle averaging method is proposed to balance the accuracy and safety issues; third, the independently estimated contact states are fused in a hidden Markov model to remove the abnormal contact state observations; finally, the object poses are refined by averaging the contact state consistent particles. Findings The experiments are performed to evaluate the effectiveness of the proposed methods. The results show that the method can achieve smooth and accurate pose estimation results and the estimated contact states are consistent with ground truth. Originality/value This paper proposes a method to obtain contact-consistent poses and contact states of objects using only visual sensors. The method tries to recover the true contact state from inaccurate visual information by fusing contact simulations results and contact consistency assumptions. The method can be used to extract pose and contact information from object manipulation tasks by just observing the demonstration, which can provide a new way for the robot to learn complex manipulation tasks.