A Hierarchical Dual-Memory Learning Model for Human Skeleton Action Recognition

Abstract

Due to many applications such as video surveillance, human machines interaction and video recovery, human actions recognition was a significant topic in computer vision. This paper proposes a self-organizing recurrent incremental network (SORIN) for human action recognition using human skeleton information. The proposed method models human working memory and episodic memory and comprises two layers of adaptive recurrent Growing-When-Required (ar-GWR) network that connected hierarchically. The working memory layer continually learns incoming perception information and encodes the learned knowledge as neurons. Similarly, the episodic memory layer further learns the spatiotemporal relationship of neurons from working memory as episode neurons to realize human actions incrementally. The proposed method integrates with OpenPose framework for human skeleton action recognition and it is validated through a series of experiments.