Lightweight emotion analysis solution using tiny machine learning for portable devices

Abstract

Deep learning-based models have obtained great improvements in facial expression recognition (FER). However, these deep models have high computational complexity and more memory during training and inference, limiting their scalability in deploying on portable devices. In addition, the exploration of the intrinsic connection between facial muscle movements and expressions has always been a huge challenge. To resolve these dilemmas, we propose an effective binary tiny machine learning (TinyML) model by combining two different attention mechanisms and binary operations. Specifically, to exploit the muscle movements in different facial expressions, we propose an effective lightweight deep model by introducing channel and spatial attention mechanisms in which learning weights for different regions can enable the network to focus on regions associated with facial expressions. Moreover, we introduce the scale factor-based binary operation to improve the inference speed. Extensive experiments on three public facial expression datasets prove that our proposed model can achieve advanced performance with 70 K parameters and 0.96MB model size. We have ported and tested our model on the Seeed XIAO ESP32S3 Sense platform, showing the superiority of what was proposed in terms of inference speed.