Multi-channel features fitted 3D CNNs and LSTMs for human activity recognition

Abstract

Human activity recognition has been widely used in many fields, especially in video surveillance and virtual reality, etc. The paper investigates a general feature combination method for a relatively new 3D CNNs and LSTMs fusion model in human activity recognition. All the features used in this combination method are from human activity videos without manually extracting features or any prior knowledge, and the model has good generalization performance. Through extracting multi-channel features of the motion optical flow vector, grayscale and body edge, putting them to 3D convolutional neural network, and processing time characteristics within Long-Short Term Memory neural network, the recognition rate of the model rises greatly. The experiment selects KTH dataset as the data source. The model based on RGB is used to compare with the model based on multi-channel features. It shows that multi-channel features can improve recognition accuracy rate obviously, and have great robustness in different scenes, which proves that it is an efficient feature combination method fitted 3D CNNs and LSTMs.