Surface electromyography (sEMG), a human-machine interface for gesture recognition, has shown promising potential for decoding motor intentions, but a variety of nonideal factors restrict its practical application in assistive robots. In this paper, we summarized the current mainstream gesture recognition strategies and proposed a gesture recognition method based on multimodal canonical correlation analysis feature fusion classification (MCAFC) for a nonideal condition that occurs in daily life, i.e., posture variations. The deep features of the sEMG and acceleration signals were first extracted via convolutional neural networks. A canonical correlation analysis was subsequently performed to associate the deep features of the two modalities. The transformed features were utilized as inputs to a linear discriminant analysis classifier to recognize the corresponding gestures. Both offline and real-time experiments were conducted on eight non-disabled subjects. The experimental results indicated that MCAFC achieved an average classification accuracy, average motion completion rate, and average motion completion time of 93.44%, 94.05%, and 1.38 s, respectively, with multiple dynamic postures, indicating significantly better performance than that of comparable methods. The results demonstrate the feasibility and superiority of the proposed multimodal signal feature fusion method for gesture recognition with posture variations, providing a new scheme for myoelectric control.
{"title":"Improving Hand Gesture Recognition Robustness to Dynamic Posture Variations by Multimodal Deep Feature Fusion","authors":"Jiwei Li;Bi Zhang;Wanxin Chen;Chunguang Bu;Yiwen Zhao;Xingang Zhao","doi":"10.1109/TNSRE.2024.3447669","DOIUrl":"10.1109/TNSRE.2024.3447669","url":null,"abstract":"Surface electromyography (sEMG), a human-machine interface for gesture recognition, has shown promising potential for decoding motor intentions, but a variety of nonideal factors restrict its practical application in assistive robots. In this paper, we summarized the current mainstream gesture recognition strategies and proposed a gesture recognition method based on multimodal canonical correlation analysis feature fusion classification (MCAFC) for a nonideal condition that occurs in daily life, i.e., posture variations. The deep features of the sEMG and acceleration signals were first extracted via convolutional neural networks. A canonical correlation analysis was subsequently performed to associate the deep features of the two modalities. The transformed features were utilized as inputs to a linear discriminant analysis classifier to recognize the corresponding gestures. Both offline and real-time experiments were conducted on eight non-disabled subjects. The experimental results indicated that MCAFC achieved an average classification accuracy, average motion completion rate, and average motion completion time of 93.44%, 94.05%, and 1.38 s, respectively, with multiple dynamic postures, indicating significantly better performance than that of comparable methods. The results demonstrate the feasibility and superiority of the proposed multimodal signal feature fusion method for gesture recognition with posture variations, providing a new scheme for myoelectric control.","PeriodicalId":13419,"journal":{"name":"IEEE Transactions on Neural Systems and Rehabilitation Engineering","volume":"32 ","pages":"3104-3115"},"PeriodicalIF":4.8,"publicationDate":"2024-08-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10643564","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142035752","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Facial palsy evaluation (FPE) aims to assess facial palsy severity of patients, which plays a vital role in facial functional treatment and rehabilitation. The traditional manners of FPE are based on subjective judgment by clinicians, which may ultimately depend on individual experience. Compared with subjective and manual evaluation, objective and automated evaluation using artificial intelligence (AI) has shown great promise in improving traditional manners and recently received significant attention. The motivation of this survey paper is mainly to provide a systemic review that would guide researchers in conducting their future research work and thus make automatic FPE applicable in real-life situations. In this survey, we comprehensively review the state-of-the-art development of AI-based FPE. First, we summarize the general pipeline of FPE systems with the related background introduction. Following this pipeline, we introduce the existing public databases and give the widely used objective evaluation metrics of FPE. In addition, the preprocessing methods in FPE are described. Then, we provide an overview of selected key publications from 2008 and summarize the state-of-the-art methods of FPE that are designed based on AI techniques. Finally, we extensively discuss the current research challenges faced by FPE and provide insights about potential future directions for advancing state-of-the-art research in this field.
{"title":"Artificial Intelligence-Based Facial Palsy Evaluation: A Survey","authors":"Yating Zhang;Weixiang Gao;Hui Yu;Junyu Dong;Yifan Xia","doi":"10.1109/TNSRE.2024.3447881","DOIUrl":"10.1109/TNSRE.2024.3447881","url":null,"abstract":"Facial palsy evaluation (FPE) aims to assess facial palsy severity of patients, which plays a vital role in facial functional treatment and rehabilitation. The traditional manners of FPE are based on subjective judgment by clinicians, which may ultimately depend on individual experience. Compared with subjective and manual evaluation, objective and automated evaluation using artificial intelligence (AI) has shown great promise in improving traditional manners and recently received significant attention. The motivation of this survey paper is mainly to provide a systemic review that would guide researchers in conducting their future research work and thus make automatic FPE applicable in real-life situations. In this survey, we comprehensively review the state-of-the-art development of AI-based FPE. First, we summarize the general pipeline of FPE systems with the related background introduction. Following this pipeline, we introduce the existing public databases and give the widely used objective evaluation metrics of FPE. In addition, the preprocessing methods in FPE are described. Then, we provide an overview of selected key publications from 2008 and summarize the state-of-the-art methods of FPE that are designed based on AI techniques. Finally, we extensively discuss the current research challenges faced by FPE and provide insights about potential future directions for advancing state-of-the-art research in this field.","PeriodicalId":13419,"journal":{"name":"IEEE Transactions on Neural Systems and Rehabilitation Engineering","volume":"32 ","pages":"3116-3134"},"PeriodicalIF":4.8,"publicationDate":"2024-08-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10643562","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142035751","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Precise control of strength is of significant importance in upper limb functional rehabilitation. Understanding the neuro-muscular response in strength regulation can help optimize the rehabilitation prescriptions and facilitate the relative training process for recovery control. This study aimed to investigate the inherent characteristics of neural-muscular activity during dynamic hand strength adjustment. Four dynamic grip force tracking modes were set by manipulating different magnitude and speed of force variations, and thirteen healthy young individuals took participation in the experiment. Electroencephalography were recorded in the contralateral sensorimotor cortex area, as well as the electromyography from the first dorsal interosseous muscle were collected synchronously. The metrics of the Event-related desynchronization, the electromyography stability index, and the force variation, were used to represent the corresponding cortical neural responses, muscle contraction activities, and the level of strength regulation, respectively; and further neuro-muscular coupling between the sensorimotor cortex and the first dorsal interosseous muscle was investigated by transfer entropy analysis. The results indicated a strong relationship that the increase of force regulation demand would result in a force variation increase as well as a stability reduction in muscle motor unit output. Meanwhile, the intensity of neural response increased in both the $alpha $ and $beta $ frequency bands. As the force regulation demand increased, the strength of bidirectional transfer entropy showed a clear shift from $beta $ to the $gamma $ frequency band, which facilitate rapid integration of dynamic strength compensation to adapt to motor task changes.
{"title":"Neuro-Muscular Responses Adaptation to Dynamic Changes in Grip Strength","authors":"Bowen Xiao;Limeng Liu;Lin Chen;Xing Wang;Xin Zhang;Xiaoyu Liu;Wensheng Hou;Xiaoying Wu","doi":"10.1109/TNSRE.2024.3447062","DOIUrl":"10.1109/TNSRE.2024.3447062","url":null,"abstract":"Precise control of strength is of significant importance in upper limb functional rehabilitation. Understanding the neuro-muscular response in strength regulation can help optimize the rehabilitation prescriptions and facilitate the relative training process for recovery control. This study aimed to investigate the inherent characteristics of neural-muscular activity during dynamic hand strength adjustment. Four dynamic grip force tracking modes were set by manipulating different magnitude and speed of force variations, and thirteen healthy young individuals took participation in the experiment. Electroencephalography were recorded in the contralateral sensorimotor cortex area, as well as the electromyography from the first dorsal interosseous muscle were collected synchronously. The metrics of the Event-related desynchronization, the electromyography stability index, and the force variation, were used to represent the corresponding cortical neural responses, muscle contraction activities, and the level of strength regulation, respectively; and further neuro-muscular coupling between the sensorimotor cortex and the first dorsal interosseous muscle was investigated by transfer entropy analysis. The results indicated a strong relationship that the increase of force regulation demand would result in a force variation increase as well as a stability reduction in muscle motor unit output. Meanwhile, the intensity of neural response increased in both the \u0000<inline-formula> <tex-math>$alpha $ </tex-math></inline-formula>\u0000 and \u0000<inline-formula> <tex-math>$beta $ </tex-math></inline-formula>\u0000 frequency bands. As the force regulation demand increased, the strength of bidirectional transfer entropy showed a clear shift from \u0000<inline-formula> <tex-math>$beta $ </tex-math></inline-formula>\u0000 to the \u0000<inline-formula> <tex-math>$gamma $ </tex-math></inline-formula>\u0000 frequency band, which facilitate rapid integration of dynamic strength compensation to adapt to motor task changes.","PeriodicalId":13419,"journal":{"name":"IEEE Transactions on Neural Systems and Rehabilitation Engineering","volume":"32 ","pages":"3189-3198"},"PeriodicalIF":4.8,"publicationDate":"2024-08-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10643225","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142017383","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The daily experience of mental stress profoundly influences our health and work performance while concurrently triggering alterations in brain electrical activity. Electroencephalogram (EEG) is a widely adopted method for assessing cognitive and affective states. This study delves into the EEG correlates of stress and the potential use of resting EEG in evaluating stress levels. Over 13 weeks, our longitudinal study focuses on the real-life experiences of college students, collecting data from each of the 18 participants across multiple days in classroom settings. To tackle the complexity arising from the multitude of EEG features and the imbalance in data samples across stress levels, we use the sequential backward selection (SBS) method for feature selection and the adaptive synthetic (ADASYN) sampling algorithm for imbalanced data. Our findings unveil that delta and theta features account for approximately 50% of the selected features through the SBS process. In leave-one-out (LOO) cross-validation, the combination of band power and pair-wise coherence (COH) achieves a maximum balanced accuracy of 94.8% in stress-level detection for the above daily stress dataset. Notably, using ADASYN and borderline synthesized minority over-sampling technique (borderline-SMOTE) methods enhances model accuracy compared to the traditional SMOTE approach. These results provide valuable insights into using EEG signals for assessing stress levels in real-life scenarios, shedding light on potential strategies for managing stress more effectively.
{"title":"Accurate Mental Stress Detection Using Sequential Backward Selection and Adaptive Synthetic Methods","authors":"Hui-Chun Tseng;Kuang-Yi Tai;Yu-Zheng Ma;Lan-Da Van;Li-Wei Ko;Tzyy-Ping Jung","doi":"10.1109/TNSRE.2024.3447274","DOIUrl":"10.1109/TNSRE.2024.3447274","url":null,"abstract":"The daily experience of mental stress profoundly influences our health and work performance while concurrently triggering alterations in brain electrical activity. Electroencephalogram (EEG) is a widely adopted method for assessing cognitive and affective states. This study delves into the EEG correlates of stress and the potential use of resting EEG in evaluating stress levels. Over 13 weeks, our longitudinal study focuses on the real-life experiences of college students, collecting data from each of the 18 participants across multiple days in classroom settings. To tackle the complexity arising from the multitude of EEG features and the imbalance in data samples across stress levels, we use the sequential backward selection (SBS) method for feature selection and the adaptive synthetic (ADASYN) sampling algorithm for imbalanced data. Our findings unveil that delta and theta features account for approximately 50% of the selected features through the SBS process. In leave-one-out (LOO) cross-validation, the combination of band power and pair-wise coherence (COH) achieves a maximum balanced accuracy of 94.8% in stress-level detection for the above daily stress dataset. Notably, using ADASYN and borderline synthesized minority over-sampling technique (borderline-SMOTE) methods enhances model accuracy compared to the traditional SMOTE approach. These results provide valuable insights into using EEG signals for assessing stress levels in real-life scenarios, shedding light on potential strategies for managing stress more effectively.","PeriodicalId":13419,"journal":{"name":"IEEE Transactions on Neural Systems and Rehabilitation Engineering","volume":"32 ","pages":"3095-3103"},"PeriodicalIF":4.8,"publicationDate":"2024-08-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10643212","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142017382","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-20DOI: 10.1109/TNSRE.2024.3446588
Zhengwang Xia;Tao Zhou;Zhuqing Jiao;Jianfeng Lu
Brain networks/graphs have been widely recognized as powerful and efficient tools for identifying neurological disorders. In recent years, various graph neural network models have been developed to automatically extract features from brain networks. However, a key limitation of these models is that the inputs, namely brain networks/graphs, are constructed using predefined statistical metrics (e.g., Pearson correlation) and are not learnable. The lack of learnability restricts the flexibility of these approaches. While statistically-specific brain networks can be highly effective in recognizing certain diseases, their performance may not exhibit robustness when applied to other types of brain disorders. To address this issue, we propose a novel module called Brain Structure Inference (termed BSI), which can be seamlessly integrated with multiple downstream tasks within a unified framework, enabling end-to-end training. It is highly flexible to learn the most beneficial underlying graph structures directly for specific downstream tasks. The proposed method achieves classification accuracies of 74.83% and 79.18% on two publicly available datasets, respectively. This suggests an improvement of at least 3% over the best-performing existing methods for both tasks. In addition to its excellent performance, the proposed method is highly interpretable, and the results are generally consistent with previous findings.
{"title":"Learnable Brain Connectivity Structures for Identifying Neurological Disorders","authors":"Zhengwang Xia;Tao Zhou;Zhuqing Jiao;Jianfeng Lu","doi":"10.1109/TNSRE.2024.3446588","DOIUrl":"10.1109/TNSRE.2024.3446588","url":null,"abstract":"Brain networks/graphs have been widely recognized as powerful and efficient tools for identifying neurological disorders. In recent years, various graph neural network models have been developed to automatically extract features from brain networks. However, a key limitation of these models is that the inputs, namely brain networks/graphs, are constructed using predefined statistical metrics (e.g., Pearson correlation) and are not learnable. The lack of learnability restricts the flexibility of these approaches. While statistically-specific brain networks can be highly effective in recognizing certain diseases, their performance may not exhibit robustness when applied to other types of brain disorders. To address this issue, we propose a novel module called Brain Structure Inference (termed BSI), which can be seamlessly integrated with multiple downstream tasks within a unified framework, enabling end-to-end training. It is highly flexible to learn the most beneficial underlying graph structures directly for specific downstream tasks. The proposed method achieves classification accuracies of 74.83% and 79.18% on two publicly available datasets, respectively. This suggests an improvement of at least 3% over the best-performing existing methods for both tasks. In addition to its excellent performance, the proposed method is highly interpretable, and the results are generally consistent with previous findings.","PeriodicalId":13419,"journal":{"name":"IEEE Transactions on Neural Systems and Rehabilitation Engineering","volume":"32 ","pages":"3084-3094"},"PeriodicalIF":4.8,"publicationDate":"2024-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10640116","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142008759","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The deficit in social interaction skills among individuals with autism spectrum disorder (ASD) is strongly influenced by personal experiences and social environments. Neuroimaging studies have previously highlighted the link between social impairment and brain activity in ASD. This study aims to develop a method for assessing and identifying ASD using a social cognitive game-based paradigm combined with electroencephalo-graphy (EEG) signaling features. Typically developing (TD) participants and autistic preadolescents and teenagers were recruited to participate in a social game while 12-channel EEG signals were recorded. The EEG signals underwent preprocessing to analyze local brain activities, including event-related potentials (ERPs) and time-frequency features. Additionally, the global brain network’s functional connectivity between brain regions was evaluated using phase-lag indices (PLIs). Subsequently, machine learning models were employed to assess the neurophysiological features. Results indicated pronounced ERP components, particularly the late positive potential (LPP), in parietal regions during social training. Autistic preadolescents and teenagers exhibited lower LPP amplitudes and larger P200 amplitudes compared to TD participants. Reduced theta synchronization was also observed in the ASD group. Aberrant functional connectivity within certain time intervals was noted in the ASD group. Machine learning analysis revealed that support-vector machines achieved a sensitivity of 100%, specificity of 91.7%, and accuracy of 95.8% as part of the performance evaluation when utilizing ERP and brain oscillation features for ASD characterization. These findings suggest that social interaction difficulties in autism are linked to specific brain activation patterns. Traditional behavioral assessments face challenges of subjectivity and accuracy, indicating the potential use of social training interfaces and EEG features for cognitive assessment in ASD.
{"title":"Characterizing Autism Spectrum Disorder Through Fusion of Local Cortical Activation and Global Functional Connectivity Using Game-Based Stimuli and a Mobile EEG System","authors":"Yi-Li Tseng;Chia-Hsin Lee;Yen-Nan Chiu;Wen-Che Tsai;Jui-Sheng Wang;Wei-Chen Wu;Yi-Ling Chien","doi":"10.1109/TNSRE.2024.3417210","DOIUrl":"10.1109/TNSRE.2024.3417210","url":null,"abstract":"The deficit in social interaction skills among individuals with autism spectrum disorder (ASD) is strongly influenced by personal experiences and social environments. Neuroimaging studies have previously highlighted the link between social impairment and brain activity in ASD. This study aims to develop a method for assessing and identifying ASD using a social cognitive game-based paradigm combined with electroencephalo-graphy (EEG) signaling features. Typically developing (TD) participants and autistic preadolescents and teenagers were recruited to participate in a social game while 12-channel EEG signals were recorded. The EEG signals underwent preprocessing to analyze local brain activities, including event-related potentials (ERPs) and time-frequency features. Additionally, the global brain network’s functional connectivity between brain regions was evaluated using phase-lag indices (PLIs). Subsequently, machine learning models were employed to assess the neurophysiological features. Results indicated pronounced ERP components, particularly the late positive potential (LPP), in parietal regions during social training. Autistic preadolescents and teenagers exhibited lower LPP amplitudes and larger P200 amplitudes compared to TD participants. Reduced theta synchronization was also observed in the ASD group. Aberrant functional connectivity within certain time intervals was noted in the ASD group. Machine learning analysis revealed that support-vector machines achieved a sensitivity of 100%, specificity of 91.7%, and accuracy of 95.8% as part of the performance evaluation when utilizing ERP and brain oscillation features for ASD characterization. These findings suggest that social interaction difficulties in autism are linked to specific brain activation patterns. Traditional behavioral assessments face challenges of subjectivity and accuracy, indicating the potential use of social training interfaces and EEG features for cognitive assessment in ASD.","PeriodicalId":13419,"journal":{"name":"IEEE Transactions on Neural Systems and Rehabilitation Engineering","volume":"32 ","pages":"3026-3035"},"PeriodicalIF":4.8,"publicationDate":"2024-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10639538","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142008758","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Auditory Brainstem Response (ABR) is an evoked potential in the brainstem’s neural centers in response to sound stimuli. Clinically, characteristic waves, especially Wave V latency, extracted from ABR can objectively indicate auditory loss and diagnose diseases. Several methods have been developed for the extraction of characteristic waves. To ensure the effectiveness of the method, most of the methods are time-consuming and rely on the heavy workloads of clinicians. To reduce the workload of clinicians, automated extraction methods have been developed. However, the above methods also have limitations. This study introduces a novel deep learning network for automatic extraction of Wave V latency, named ABR-Attention. ABR-Attention model includes a self-attention module, first and second-derivative attention module, and regressor module. Experiments are conducted on the accuracy with 10-fold cross-validation, the effects on different sound pressure levels (SPLs), the effects of different error scales and the effects of ablation. ABR-Attention shows efficacy in extracting Wave V latency of ABR, with an overall accuracy of $96.76~pm ~0.41$ % and an error scale of 0.1ms, and provides a new solution for objective localization of ABR characteristic waves.
听性脑干反应（ABR）是脑干神经中枢对声音刺激做出反应的一种诱发电位。在临床上，从 ABR 中提取的特征波（尤其是 V 波潜伏期）可以客观地显示听觉损失和诊断疾病。目前已开发出多种提取特征波的方法。为了确保方法的有效性，大多数方法都很耗时，并且依赖于临床医生繁重的工作量。为了减轻临床医生的工作量，人们开发了自动提取方法。然而，上述方法也存在局限性。本研究介绍了一种用于自动提取波 V 潜伏期的新型深度学习网络，名为 ABR-Attention。ABR-Attention 模型包括自我注意模块、第一和第二派生注意模块以及回归模块。实验内容包括 10 倍交叉验证的准确性、对不同声压级 (SPL) 的影响、不同误差标度的影响以及消融的影响。ABR-Attention 在提取 ABR 第 V 波潜伏期方面效果显著，总体准确率为 96.76±0.41%，误差范围为 0.1ms，为客观定位 ABR 特征波提供了新的解决方案。
{"title":"ABR-Attention: An Attention-Based Model for Precisely Localizing Auditory Brainstem Response","authors":"Junyu Ji;Xin Wang;Xiaobei Jing;Mingxing Zhu;Hongguang Pan;Desheng Jia;Chunrui Zhao;Xu Yong;Yangjie Xu;Guoru Zhao;Poly Z.H. Sun;Guanglin Li;Shixiong Chen","doi":"10.1109/TNSRE.2024.3445936","DOIUrl":"10.1109/TNSRE.2024.3445936","url":null,"abstract":"Auditory Brainstem Response (ABR) is an evoked potential in the brainstem’s neural centers in response to sound stimuli. Clinically, characteristic waves, especially Wave V latency, extracted from ABR can objectively indicate auditory loss and diagnose diseases. Several methods have been developed for the extraction of characteristic waves. To ensure the effectiveness of the method, most of the methods are time-consuming and rely on the heavy workloads of clinicians. To reduce the workload of clinicians, automated extraction methods have been developed. However, the above methods also have limitations. This study introduces a novel deep learning network for automatic extraction of Wave V latency, named ABR-Attention. ABR-Attention model includes a self-attention module, first and second-derivative attention module, and regressor module. Experiments are conducted on the accuracy with 10-fold cross-validation, the effects on different sound pressure levels (SPLs), the effects of different error scales and the effects of ablation. ABR-Attention shows efficacy in extracting Wave V latency of ABR, with an overall accuracy of \u0000<inline-formula> <tex-math>$96.76~pm ~0.41$ </tex-math></inline-formula>\u0000% and an error scale of 0.1ms, and provides a new solution for objective localization of ABR characteristic waves.","PeriodicalId":13419,"journal":{"name":"IEEE Transactions on Neural Systems and Rehabilitation Engineering","volume":"32 ","pages":"3179-3188"},"PeriodicalIF":4.8,"publicationDate":"2024-08-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10639446","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142004163","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-16DOI: 10.1109/TNSRE.2024.3444890
Kwangsub Song;Hyung Eun Shin;Wookhyun Park;Daehyun Lee;Jaeyoung Jang;Ga Yang Shim;Sangui Choi;Miji Kim;Hooman Lee;Chang Won Won
Sarcopenia is a comprehensive degenerative disease with the progressive loss of skeletal muscle mass with age, accompanied by the loss of muscle strength and muscle dysfunction. Individuals with unmanaged sarcopenia may experience adverse outcomes. Periodically monitoring muscle function to detect muscle degeneration caused by sarcopenia and treating degenerated muscles is essential. We proposed a digital biomarker measurement technique using surface electromyography (sEMG) with electrical stimulation and wearable device to conveniently monitor muscle function at home. When motor neurons and muscle fibers are electrically stimulated, stimulated muscle contraction signals (SMCSs) can be obtained using an sEMG sensor. As motor neuron activation is important for muscle contraction and strength, their action potentials for electrical stimulation represent the muscle function. Thus, the SMCSs are closely related to muscle function, presumptively. Using the SMCSs data, a feature vector concatenating spectrogram-based features and deep learning features extracted from a convolutional neural network model using continuous wavelet transform images was used as the input to train a regression model for measuring the digital biomarker. To verify muscle function measurement technique, we recruited 98 healthy participants aged 20–60 years including 48 [49%] men who volunteered for this study. The Pearson correlation coefficient between the label and model estimates was 0.89, suggesting that the proposed model can robustly estimate the label using SMCSs, with mean error and standard deviation of -0.06 and 0.68, respectively. In conclusion, measuring muscle function using the proposed system that involves SMCSs is feasible.
{"title":"Digital Biomarker for Muscle Function Assessment Using Surface Electromyography With Electrical Stimulation and a Non-Invasive Wearable Device","authors":"Kwangsub Song;Hyung Eun Shin;Wookhyun Park;Daehyun Lee;Jaeyoung Jang;Ga Yang Shim;Sangui Choi;Miji Kim;Hooman Lee;Chang Won Won","doi":"10.1109/TNSRE.2024.3444890","DOIUrl":"10.1109/TNSRE.2024.3444890","url":null,"abstract":"Sarcopenia is a comprehensive degenerative disease with the progressive loss of skeletal muscle mass with age, accompanied by the loss of muscle strength and muscle dysfunction. Individuals with unmanaged sarcopenia may experience adverse outcomes. Periodically monitoring muscle function to detect muscle degeneration caused by sarcopenia and treating degenerated muscles is essential. We proposed a digital biomarker measurement technique using surface electromyography (sEMG) with electrical stimulation and wearable device to conveniently monitor muscle function at home. When motor neurons and muscle fibers are electrically stimulated, stimulated muscle contraction signals (SMCSs) can be obtained using an sEMG sensor. As motor neuron activation is important for muscle contraction and strength, their action potentials for electrical stimulation represent the muscle function. Thus, the SMCSs are closely related to muscle function, presumptively. Using the SMCSs data, a feature vector concatenating spectrogram-based features and deep learning features extracted from a convolutional neural network model using continuous wavelet transform images was used as the input to train a regression model for measuring the digital biomarker. To verify muscle function measurement technique, we recruited 98 healthy participants aged 20–60 years including 48 [49%] men who volunteered for this study. The Pearson correlation coefficient between the label and model estimates was 0.89, suggesting that the proposed model can robustly estimate the label using SMCSs, with mean error and standard deviation of -0.06 and 0.68, respectively. In conclusion, measuring muscle function using the proposed system that involves SMCSs is feasible.","PeriodicalId":13419,"journal":{"name":"IEEE Transactions on Neural Systems and Rehabilitation Engineering","volume":"32 ","pages":"3048-3058"},"PeriodicalIF":4.8,"publicationDate":"2024-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10638087","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141992291","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-16DOI: 10.1109/TNSRE.2024.3445115
Hanrui Wu;Zhengyan Ma;Zhenpeng Guo;Yanxin Wu;Jia Zhang;Guoxu Zhou;Jinyi Long
Electroencephalogram (EEG) signals play an important role in brain-computer interface (BCI) applications. Recent studies have utilized transfer learning to assist the learning task in the new subject, i.e., target domain, by leveraging beneficial information from previous subjects, i.e., source domains. Nevertheless, EEG signals involve sensitive personal mental and health information. Thus, privacy concern becomes a critical issue. In addition, existing methods mostly assume that a portion of the new subject’s data is available and perform alignment or adaptation between the source and target domains. However, in some practical scenarios, new subjects prefer prompt BCI utilization over the time-consuming process of collecting data for calibration and adaptation, which makes the above assumption difficult to hold. To address the above challenges, we propose Online Source-Free Transfer Learning (OSFTL) for privacy-preserving EEG classification. Specifically, the learning procedure contains offline and online stages. At the offline stage, multiple model parameters are obtained based on the EEG samples from multiple source subjects. OSFTL only needs access to these source model parameters to preserve the privacy of the source subjects. At the online stage, a target classifier is trained based on the online sequence of EEG instances. Subsequently, OSFTL learns a weighted combination of the source and target classifiers to obtain the final prediction for each target instance. Moreover, to ensure good transferability, OSFTL dynamically updates the transferred weight of each source domain based on the similarity between each source classifier and the target classifier. Comprehensive experiments on both simulated and real-world applications demonstrate the effectiveness of the proposed method, indicating the potential of OSFTL to facilitate the deployment of BCI applications outside of controlled laboratory settings.
{"title":"Online Privacy-Preserving EEG Classification by Source-Free Transfer Learning","authors":"Hanrui Wu;Zhengyan Ma;Zhenpeng Guo;Yanxin Wu;Jia Zhang;Guoxu Zhou;Jinyi Long","doi":"10.1109/TNSRE.2024.3445115","DOIUrl":"10.1109/TNSRE.2024.3445115","url":null,"abstract":"Electroencephalogram (EEG) signals play an important role in brain-computer interface (BCI) applications. Recent studies have utilized transfer learning to assist the learning task in the new subject, i.e., target domain, by leveraging beneficial information from previous subjects, i.e., source domains. Nevertheless, EEG signals involve sensitive personal mental and health information. Thus, privacy concern becomes a critical issue. In addition, existing methods mostly assume that a portion of the new subject’s data is available and perform alignment or adaptation between the source and target domains. However, in some practical scenarios, new subjects prefer prompt BCI utilization over the time-consuming process of collecting data for calibration and adaptation, which makes the above assumption difficult to hold. To address the above challenges, we propose Online Source-Free Transfer Learning (OSFTL) for privacy-preserving EEG classification. Specifically, the learning procedure contains offline and online stages. At the offline stage, multiple model parameters are obtained based on the EEG samples from multiple source subjects. OSFTL only needs access to these source model parameters to preserve the privacy of the source subjects. At the online stage, a target classifier is trained based on the online sequence of EEG instances. Subsequently, OSFTL learns a weighted combination of the source and target classifiers to obtain the final prediction for each target instance. Moreover, to ensure good transferability, OSFTL dynamically updates the transferred weight of each source domain based on the similarity between each source classifier and the target classifier. Comprehensive experiments on both simulated and real-world applications demonstrate the effectiveness of the proposed method, indicating the potential of OSFTL to facilitate the deployment of BCI applications outside of controlled laboratory settings.","PeriodicalId":13419,"journal":{"name":"IEEE Transactions on Neural Systems and Rehabilitation Engineering","volume":"32 ","pages":"3059-3070"},"PeriodicalIF":4.8,"publicationDate":"2024-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10638104","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141992292","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-08-16DOI: 10.1109/TNSRE.2024.3444461
Lingchao Xie;Chung-Hwi Yi;Oh-Yun Kwon;Woochol Joseph Choi;Ji-Hyuk Park;Sanghyun Cho
This study aimed to evaluate the effectiveness of our newly developed virtual reality head-mounted display (VR-HMD) “walker avoidance” game in reducing step-aside reaction time (SART) and enhancing agility in collision avoidance. Fifteen young adults in experimental group (EG) engaged in the “walker avoidance” game, while another 15 young adults in the control group (CG) played the “first touch” tutorial. The results showed the EG had significant decreases (p < 0.01) in both SART-standing and SART-walking when compared with pre-intervention measurements. Compared with the CG, the EG SART-standing exhibited significant decreases in both the first (p = 0.001) and second (p < 0.001) measurements post-intervention; the EG SART-walking demonstrated significant decreases in all (p < 0.05) measurements, except for pre-intervention measurement. One-dimensional statistical parametric mapping (spm1d) also demonstrated significant differences in most of the electromyography and forefoot/hindfoot ground reaction force results because the step-aside movement became quicker in the EG following training. After pushing the leg-heel contact, the EG participants made a toe-off sooner than the CG participants. Following two sessions of our newly developed “walker avoidance” game, conducted 1 week apart, the EG exhibited less collisions with virtual pedestrians and reduced reaction times to unpredictable directional change measurements compared with the CG. This study demonstrated the effectiveness of this targeted VR training program in improving motor function, which introduced a novel approach to rehabilitation as a digital therapy. It offers innovative perspectives and an approach for clinical rehabilitation, while also providing new ideas for the VR content development industry.
{"title":"The Effects of a VR Training Program for Walker Avoidance Skill Improvement: A Randomized Controlled Trial","authors":"Lingchao Xie;Chung-Hwi Yi;Oh-Yun Kwon;Woochol Joseph Choi;Ji-Hyuk Park;Sanghyun Cho","doi":"10.1109/TNSRE.2024.3444461","DOIUrl":"10.1109/TNSRE.2024.3444461","url":null,"abstract":"This study aimed to evaluate the effectiveness of our newly developed virtual reality head-mounted display (VR-HMD) “walker avoidance” game in reducing step-aside reaction time (SART) and enhancing agility in collision avoidance. Fifteen young adults in experimental group (EG) engaged in the “walker avoidance” game, while another 15 young adults in the control group (CG) played the “first touch” tutorial. The results showed the EG had significant decreases (p < 0.01) in both SART-standing and SART-walking when compared with pre-intervention measurements. Compared with the CG, the EG SART-standing exhibited significant decreases in both the first (p = 0.001) and second (p < 0.001) measurements post-intervention; the EG SART-walking demonstrated significant decreases in all (p < 0.05) measurements, except for pre-intervention measurement. One-dimensional statistical parametric mapping (spm1d) also demonstrated significant differences in most of the electromyography and forefoot/hindfoot ground reaction force results because the step-aside movement became quicker in the EG following training. After pushing the leg-heel contact, the EG participants made a toe-off sooner than the CG participants. Following two sessions of our newly developed “walker avoidance” game, conducted 1 week apart, the EG exhibited less collisions with virtual pedestrians and reduced reaction times to unpredictable directional change measurements compared with the CG. This study demonstrated the effectiveness of this targeted VR training program in improving motor function, which introduced a novel approach to rehabilitation as a digital therapy. It offers innovative perspectives and an approach for clinical rehabilitation, while also providing new ideas for the VR content development industry.","PeriodicalId":13419,"journal":{"name":"IEEE Transactions on Neural Systems and Rehabilitation Engineering","volume":"32 ","pages":"3018-3025"},"PeriodicalIF":4.8,"publicationDate":"2024-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10637447","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141992293","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：