Hand Detection and Hand Recognition Application Design for Human Computer Interaction Using SSD and Hand Landmark

Abstract

This research is intended to support computer interaction technology, especially the operation of computers to minimize the adverse effects of the dangers of direct physical contact between hands and technological objects. This research was conducted with the aim of utilizing the movement of the limbs of the hand as an object of interaction that bridges between humans and computers. With this research, it is hoped that the design results can minimize the risk of spreading bacteria and viruses that cause health problems. The objects used in the design will focus on the pattern/shape and movement of the hands, and in training the pattern/shape of the hand as a pointing tool and keyboard shortcut input. The design was built by using a webcam as a sensor to capture images. This design used the help of the field of artificial intelligence, hand detection using the Single-Shot Detector method with hand recognition using Hand Landmark. The value of success in functionality obtained from the application was 94.73%. The results of the model evaluation obtained that the final average recall value was 96%, the precision value was 100%, and the accuracy value was 96%. From the research that has been done, it is possible to replace the mouse function and several keyboard shortcuts with right- and left-hand movements.