An uncalibrated visual servo control method of manipulator for multiple peg-in-hole assembly based on projective homography

Abstract

This study proposes an uncalibrated visual servo positioning assembly algorithm based on projected homography matrix, aimed at low-cost camera-guided multiple peg-in-hole assembly. Compared with traditional schemes, the proposed method avoids the use of force sensors and deep reinforcement learning strategies, thereby reducing the interaction with the real world and the risk of damage to assemblies with soft materials, weak stiffness, and small dimensions. First, we design an assembly path constraint method to realise image feature point mapping in the lower plate by introducing a virtual image plane in the image plane, which transforms the localisation problem in the assembly into an overlapping problem between the upper image plane and the virtual plane and prevents the skewing of the traditional visual servoing method at the assembly point. Second, a new task function is designed using the elements of the projective homography matrix to realise visual servoing without the need for previous knowledge of the camera’s intrinsic parameters and hand–eye relationships. This has a lower calculation cost and better accuracy performance compared with the traditional uncalibrated visual servoing. Subsequently, a Kalman filter is introduced to evaluate the image Jacobian matrix in the task function, and a long short-term memory (LSTM) neural network is used to compensate for the image error. Through these operations, non-Gaussian noise can be estimated. Finally, the effectiveness of the method in actual environments is verified through simulations and experiments, with a 95% success rate compared with traditional vision servo assembly and a maximum localisation error of 1 pixel. This result is significant for multiple peg-in-hole assemblies in actual precision and ultraprecision manufacturing areas.