Manifold Guided Graph Neural Networks for Skeleton-based Action Recognition in Human Computer Interaction Videos

Abstract

As the key application in video analysis for human computer interaction (HCI), the problem of skeleton-based action recognition has been solved by some researchers with graph neural networks, but it remains an unsolved issue on complex variations of spatiotemporal dependence across skeleton joints flow. A newly dynamic spatio-temporal graph structure learning method, manifold guided graph neural networks (MGNN), was proposed to solve this problem. In MGNN, a novel manifold guided graph updating mechanism is built based on the baseline graph neural network to further describe the spatio-temporal dependence. With the manifold guided multi-scale skeleton graph, the proposed MGNN is further trained with two streams of joint and bone to improve the efficiency, which forms a single network seamlessly and enables it be trained in a same umbrella. Comparing with the existing methods, MGNN has been proved that it yields better performance on challenging datasets: NTU RGB+D 60 and Kinetics 400.