Edge-Cloud Collaboration for Human Activity Recognition on Multiple Subjects

Abstract

Multi-subject video analysis is one of the most important problems in the field of visual perception for human activity recognition on multiple subjects nowadays. However, multi-subject video analysis is difficult to achieve real-time performance at the edge due to the limited resources of edge devices and the high complexity of the Convolutional Neural Networks (CNN) model used in this task. The common processing method is to upload the video data to the cloud. However, due to the influence of network bandwidth, the transmission time is not fixed, and the latency cannot be guaranteed. Thus, statically deployed model configurations cannot meet some dynamically changing scenarios. To address these challenges, in this paper, we propose an edge-cloud collaboration processing system for multi-subject video stream analysis, which can dynamically configure and optimize the related configurations according to specific scenarios. Specifically, we provide an adaptive configuration optimization solution based on context awareness for edge devices with limited resources such that multi-subject video stream analysis can be processed completely at the edge. For other complex scenarios, we propose an edge-cloud collaboration method to achieve task segmentation and collaboration to meet the performance requirements of the complex scenarios. Experimental results show that our method can achieve an average accuracy of 91.3% and the latency of less than 78ms with arbitrary runtime state.