An AI-enabled self-sustaining sensing lower-limb motion detection system for HMI in the metaverse

Abstract

In light of the accelerated advancement of metaverse technologies and the growing necessity for sophisticated Human-machine interaction (HMI), the need for accurate and reliable motion detection systems has become more critical, especially in healthcare and rehabilitation. Traditional motion detection systems often face challenges such as limited portability, dependence on external power sources, and insufficient accuracy in complex environments. To address these issues, this paper presents the development of an AI-enabled self-sustaining sensing lower-limb motion detection system (SS-LMD) designed for HMI, such as rehabilitation training in the metaverse. The SS-LMD system comprises wearable TENG-based sensors that monitor lower-limb muscle movements, assess different deep learning algorithms and select an advanced real-time data processing method with a Double long short-term memory model (LSTM). The system achieves 99.8% accuracy in motion recognition and operates without external power sources, enhancing portability and user convenience. Through usability testing verified the practical application of the SS-LMD system for fitness and rehabilitation training in metaverse scenarios for users of different ages. In addition, a digital twin-based monitoring platform was developed using 5 G, database and visualization technologies to observe user status in real-time. The SS-LMD has exhibited exemplary capabilities in self-sustaining sensing and real-time capture of lower-limb motion information, thereby providing accurate motion feedback. This innovation represents a significant advance in wearable technology and holds great promise in metaverse applications in the fields of virtual fitness, smart healthcare and elderly rehabilitation.