Skeletal-based Classification for Human Activity Recognition

Abstract

Human activity recognition (HAR) is critical for determining human interactions and interpersonal relationships. Among the various classification techniques, two things become the main focus of HAR, namely the type of activity and its localization. Most of the tasks in HAR involve identifying a human scene from a series of frames in a video, where the subject being monitored is free to perform an activity. For some of the current HAR approaches, 3D sensors are used as input extractors for the skeleton/body pose of the subject being monitored. It is much more precise than using only 2D information obtained from conventional cameras. Of course, the use of 3D sensors is a significant limitation for implementing video-based surveillance systems. In this research, we use the Deep learning OpenPose 3D method as a substitute for 3D sensors that can estimate the 3D frame/pose of the subject's body identified from conventional camera 2D input sources. It is then combined with other machine learning methods for the activity classification process from the obtained 3D framework. Classifiers that can be used include Support Vector Machine (SVM), Neural Network (NN), Long short-term memory (LSTM), and Transformer. Thus, HAR can be applied flexibly in various scopes of supervision without the help of 3D sensors. The experiment results inform that Transformer is the best in accuracy while SVM is in speed.