Graph Convolutional Networks with Objects for Skeleton-Based Action Recognition

Abstract

Recently, graph convolutional neural networks has become a research hotspot for skeleton-based action recognition because of its excellent performance on graph structure data. Compared to traditional methods, it can explicitly exploit the natural connectivity among the joints and improve greater expressive power. In this paper, we propose a two-stream graph convolutional networks with objects for skeleton-based action recognition. An algorithm is designed for matching similar skeleton in adjacent frames, so that we can get the right skeletons which belong to the same person. It performs well when there are other irrelevant persons in the scene. In addition, other features are less employed except for the human joint in skeleton-based action recognition. We introduce limbs orientation information and related objects information. The related objects are treated as joint points which link with hands. The two-stream networks are built to model coordinate features and orientation features respectively, the results of two streams are fused to one. We get good results on the Kinetics dataset with our methods.