Multimodal Wearable System With Dual-Frequency Enhancement Network for Risk Recognition

Abstract

Smart wearable systems can monitor users’ physiological data in real time, detect anomalies promptly through risk recognition technologies, provide early warnings, and assist users in taking preventive measures. However, single modal information is difficult to accurately recognize the behavioral state, expression state, and environmental conditions. Furthermore, multimodal data are often affected by noise and interference, complicating the accurate identification of risky behaviors. To address these challenges, we propose a smart wearable system based on the dual-frequency enhancement network (DFENet): 1) the multimodal sensor system is designed to combine behavioral recognition, expression recognition, and environmental recognition for comprehensive monitoring and recognition of multidimensional risk factors in complex scenarios; 2) the DFENet is proposed to overcome challenges in feature extraction and accurate classification in complex environments; and 3) the behavioral recognition dataset and the expression recognition dataset are built to verify the effectiveness of the designed smart wearable system. Experimental results indicate that the proposed system can real-time achieve risk recognition across physical activity, expression state, and environmental conditions, and the proposed DFENet achieves excellent performance in accuracy, parameters, and floating-point operations (FLOPs) metrics on the three datasets. The algorithm and datasets can be downloaded at https://github.com/wtu1020/Multimodal-Wearable.