Hand keypoint detection with super resolution

Abstract

Hand interaction is an important research content in computer image processing at present. In sign language recognition, social interaction, virtual reality and augmented reality, the hand is the main input device for human interaction.Aiming at the problem of low recognition rate of hand keypoint in video, this paper proposes a deep convolutional neural network to recognize hand keypoint in video. The neural network is divided into two parts, the first part is image super-resolution, the purpose is to improve the resolution of each frame in the video, so that the image of each frame is clear, to have a high-resolution input image; The second part is the detection model, in order to ensure the real-time performance of hand keypoint detection, the model adopts a lightweight network structure to detect hand keypoint. The results show that this method has a high accuracy rate for the hand keypoint in the video, and the model was tested on the test set. Experimental results show that after adding super-resolution, the hand keypoint detection in the video is significantly improved.