Fingertip interactive tracking registration method for AR assembly system

Aiming at the problems of single input mode and lack of naturalness in the assembly process of existing AR systems, a tracking registration method of mobile AR assembly system is proposed based on multi-quantity and multi-degree of freedom natural fingertip interaction. Firstly, the real-time and stable tracking of hand area in complex environment is realized based on the hand region tracking; secondly, the fingertip detection and recognition based on K-COS and parallel vector is used to improve the precision and stability of fingertip recognition; thirdly, the special movement track of fingertip is recognized based on improved DTW algorithm, which has strong compatibility and feature gradient transformation for complex fingertip trajectory recognition; finally, through the real-time transformation of projection relationship between fingertip and virtual object, the interaction between fingertip and virtual object is made more natural and realistic. The experimental results show that in the complex environment of background, illumination, scale and rotation, the precision of fingertip detection and recognition is about 93%, and the precision of fingertip motion template matching is about 91%. The translation error of the registration method based on visual feature recognition is reduced by about 100pix compared with fingertip tracking registration method, and the efficiency of mobile AR-guided assembly method is improved by about 24.77% compared with the traditional manual assisted assembly method. These data verifies the strong interaction and practicability of the fingertips based on the user's multi-quantity and multi-degree of freedom features in the process of mobile AR guided assembly.