Edge Computing Enables Assessment of Student Community Building: An Emotion Recognition Method Based on TinyML

Abstract

Deep network-based video sentiment analysis is crucial for online evaluation tasks. However, these deep models are difficult to run on intelligent edge devices with limited computing resources. In addition, video data are susceptible to lighting interference, distortion, and background noise, which severely limits the performance of facial expression recognition. To relieve these issues, we develop an effective multi-scale semantic fusion tiny machine learning (TinyML) model based on a spatiotemporal graph convolutional network (ST-GCN) which enables robust expression recognition from facial landmark sequences. Specifically, we construct regional-connected graph data based on facial landmarks which are collected from cameras on different mobile devices. In existing spatiotemporal graph convolutional networks, we leverage the multi-scale semantic fusion mechanism to mine the hierarchical structure of facial landmarks. The experimental results on CK+ and online student community assessment sentiment analysis (OSCASA) dataset confirm that our approach yields comparable results.