Pub Date : 2024-04-17DOI: 10.1088/1741-2552/ad3b6b
Anna Sergeeva, Christian Bech Christensen, Preben Kidmose
Objective. The auditory steady-state response (ASSR) allows estimation of hearing thresholds. The ASSR can be estimated from electroencephalography (EEG) recordings from electrodes positioned on both the scalp and within the ear (ear-EEG). Ear-EEG can potentially be integrated into hearing aids, which would enable automatic fitting of the hearing device in daily life. The conventional stimuli for ASSR-based hearing assessment, such as pure tones and chirps, are monotonous and tiresome, making them inconvenient for repeated use in everyday situations. In this study we investigate the use of natural speech sounds for ASSR estimation. Approach. EEG was recorded from 22 normal hearing subjects from both scalp and ear electrodes. Subjects were stimulated monaurally with 180 min of speech stimulus modified by applying a 40 Hz amplitude modulation (AM) to an octave frequency sub-band centered at 1 kHz. Each 50 ms sub-interval in the AM sub-band was scaled to match one of 10 pre-defined levels (0–45 dB sensation level, 5 dB steps). The apparent latency for the ASSR was estimated as the maximum average cross-correlation between the envelope of the AM sub-band and the recorded EEG and was used to align the EEG signal with the audio signal. The EEG was then split up into sub-epochs of 50 ms length and sorted according to the stimulation level. ASSR was estimated for each level for both scalp- and ear-EEG. Main results. Significant ASSRs with increasing amplitude as a function of presentation level were recorded from both scalp and ear electrode configurations. Significance. Utilizing natural sounds in ASSR estimation offers the potential for electrophysiological hearing assessment that are more comfortable and less fatiguing compared to existing ASSR methods. Combined with ear-EEG, this approach may allow convenient hearing threshold estimation in everyday life, utilizing ambient sounds. Additionally, it may facilitate both initial fitting and subsequent adjustments of hearing aids outside of clinical settings.
{"title":"Towards ASSR-based hearing assessment using natural sounds","authors":"Anna Sergeeva, Christian Bech Christensen, Preben Kidmose","doi":"10.1088/1741-2552/ad3b6b","DOIUrl":"https://doi.org/10.1088/1741-2552/ad3b6b","url":null,"abstract":"<italic toggle=\"yes\">Objective</italic>. The auditory steady-state response (ASSR) allows estimation of hearing thresholds. The ASSR can be estimated from electroencephalography (EEG) recordings from electrodes positioned on both the scalp and within the ear (ear-EEG). Ear-EEG can potentially be integrated into hearing aids, which would enable automatic fitting of the hearing device in daily life. The conventional stimuli for ASSR-based hearing assessment, such as pure tones and chirps, are monotonous and tiresome, making them inconvenient for repeated use in everyday situations. In this study we investigate the use of natural speech sounds for ASSR estimation. <italic toggle=\"yes\">Approach.</italic> EEG was recorded from 22 normal hearing subjects from both scalp and ear electrodes. Subjects were stimulated monaurally with 180 min of speech stimulus modified by applying a 40 Hz amplitude modulation (AM) to an octave frequency sub-band centered at 1 kHz. Each 50 ms sub-interval in the AM sub-band was scaled to match one of 10 pre-defined levels (0–45 dB sensation level, 5 dB steps). The apparent latency for the ASSR was estimated as the maximum average cross-correlation between the envelope of the AM sub-band and the recorded EEG and was used to align the EEG signal with the audio signal. The EEG was then split up into sub-epochs of 50 ms length and sorted according to the stimulation level. ASSR was estimated for each level for both scalp- and ear-EEG. <italic toggle=\"yes\">Main results</italic>. Significant ASSRs with increasing amplitude as a function of presentation level were recorded from both scalp and ear electrode configurations. <italic toggle=\"yes\">Significance</italic>. Utilizing natural sounds in ASSR estimation offers the potential for electrophysiological hearing assessment that are more comfortable and less fatiguing compared to existing ASSR methods. Combined with ear-EEG, this approach may allow convenient hearing threshold estimation in everyday life, utilizing ambient sounds. Additionally, it may facilitate both initial fitting and subsequent adjustments of hearing aids outside of clinical settings.","PeriodicalId":16753,"journal":{"name":"Journal of neural engineering","volume":null,"pages":null},"PeriodicalIF":4.0,"publicationDate":"2024-04-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140611146","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Objective. The study of emotion recognition through electroencephalography (EEG) has garnered significant attention recently. Integrating EEG with other peripheral physiological signals may greatly enhance performance in emotion recognition. Nonetheless, existing approaches still suffer from two predominant challenges: modality heterogeneity, stemming from the diverse mechanisms across modalities, and fusion credibility, which arises when one or multiple modalities fail to provide highly credible signals. Approach. In this paper, we introduce a novel multimodal physiological signal fusion model that incorporates both intra-inter modality reconstruction and sequential pattern consistency, thereby ensuring a computable and credible EEG-based multimodal emotion recognition. For the modality heterogeneity issue, we first implement a local self-attention transformer to obtain intra-modal features for each respective modality. Subsequently, we devise a pairwise cross-attention transformer to reveal the inter-modal correlations among different modalities, thereby rendering different modalities compatible and diminishing the heterogeneity concern. For the fusion credibility issue, we introduce the concept of sequential pattern consistency to measure whether different modalities evolve in a consistent way. Specifically, we propose to measure the varying trends of different modalities, and compute the inter-modality consistency scores to ascertain fusion credibility. Main results. We conduct extensive experiments on two benchmarked datasets (DEAP and MAHNOB-HCI) with the subject-dependent paradigm. For the DEAP dataset, our method improves the accuracy by 4.58%, and the F1 score by 0.63%, compared to the state-of-the-art baseline. Similarly, for the MAHNOB-HCI dataset, our method improves the accuracy by 3.97%, and the F1 score by 4.21%. In addition, we gain much insight into the proposed framework through significance test, ablation experiments, confusion matrices and hyperparameter analysis. Consequently, we demonstrate the effectiveness of the proposed credibility modelling through statistical analysis and carefully designed experiments. Significance. All experimental results demonstrate the effectiveness of our proposed architecture and indicate that credibility modelling is essential for multimodal emotion recognition.
目的。通过脑电图（EEG）进行情绪识别的研究近来备受关注。将脑电图与其他外周生理信号整合可大大提高情绪识别的性能。然而，现有的方法仍然面临两个主要挑战：一是模态异质性，这源于不同模态之间的不同机制；二是融合可信度，当一种或多种模态无法提供高度可信的信号时，就会产生融合可信度问题。方法。在本文中，我们介绍了一种新型的多模态生理信号融合模型，该模型结合了模态内重建和序列模式一致性，从而确保了基于脑电图的多模态情绪识别的可计算性和可信度。针对模态异质性问题，我们首先实施了局部自注意变换器，以获得各模态的模态内特征。随后，我们设计了一个成对交叉注意变换器，以揭示不同模态之间的模态间相关性，从而使不同模态相互兼容，减少异质性问题。针对融合可信度问题，我们引入了序列模式一致性的概念，以衡量不同模态是否以一致的方式发展。具体来说，我们建议测量不同模态的变化趋势，并计算模态间的一致性得分，以确定融合可信度。主要结果。我们在两个基准数据集（DEAP 和 MAHNOB-HCI）上使用主体依赖范式进行了大量实验。在 DEAP 数据集上，与最先进的基线相比，我们的方法提高了 4.58% 的准确率和 0.63% 的 F1 分数。同样，对于 MAHNOB-HCI 数据集，我们的方法提高了 3.97% 的准确率和 4.21% 的 F1 分数。此外，通过显著性测试、消融实验、混淆矩阵和超参数分析，我们对所提出的框架有了更深入的了解。因此，我们通过统计分析和精心设计的实验证明了所提出的可信度建模的有效性。意义重大。所有实验结果都证明了我们提出的架构的有效性，并表明可信度建模对于多模态情感识别至关重要。
{"title":"Cross-modal credibility modelling for EEG-based multimodal emotion recognition","authors":"Yuzhe Zhang, Huan Liu, Di Wang, Dalin Zhang, Tianyu Lou, Qinghua Zheng, Chai Quek","doi":"10.1088/1741-2552/ad3987","DOIUrl":"https://doi.org/10.1088/1741-2552/ad3987","url":null,"abstract":"<italic toggle=\"yes\">Objective.</italic> The study of emotion recognition through electroencephalography (EEG) has garnered significant attention recently. Integrating EEG with other peripheral physiological signals may greatly enhance performance in emotion recognition. Nonetheless, existing approaches still suffer from two predominant challenges: modality heterogeneity, stemming from the diverse mechanisms across modalities, and fusion credibility, which arises when one or multiple modalities fail to provide highly credible signals. <italic toggle=\"yes\">Approach.</italic> In this paper, we introduce a novel multimodal physiological signal fusion model that incorporates both intra-inter modality reconstruction and sequential pattern consistency, thereby ensuring a computable and credible EEG-based multimodal emotion recognition. For the modality heterogeneity issue, we first implement a local self-attention transformer to obtain intra-modal features for each respective modality. Subsequently, we devise a pairwise cross-attention transformer to reveal the inter-modal correlations among different modalities, thereby rendering different modalities compatible and diminishing the heterogeneity concern. For the fusion credibility issue, we introduce the concept of sequential pattern consistency to measure whether different modalities evolve in a consistent way. Specifically, we propose to measure the varying trends of different modalities, and compute the inter-modality consistency scores to ascertain fusion credibility. <italic toggle=\"yes\">Main results.</italic> We conduct extensive experiments on two benchmarked datasets (DEAP and MAHNOB-HCI) with the subject-dependent paradigm. For the DEAP dataset, our method improves the accuracy by 4.58%, and the F1 score by 0.63%, compared to the state-of-the-art baseline. Similarly, for the MAHNOB-HCI dataset, our method improves the accuracy by 3.97%, and the F1 score by 4.21%. In addition, we gain much insight into the proposed framework through significance test, ablation experiments, confusion matrices and hyperparameter analysis. Consequently, we demonstrate the effectiveness of the proposed credibility modelling through statistical analysis and carefully designed experiments. <italic toggle=\"yes\">Significance.</italic> All experimental results demonstrate the effectiveness of our proposed architecture and indicate that credibility modelling is essential for multimodal emotion recognition.","PeriodicalId":16753,"journal":{"name":"Journal of neural engineering","volume":null,"pages":null},"PeriodicalIF":4.0,"publicationDate":"2024-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140611532","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-11DOI: 10.1088/1741-2552/ad39a6
Zhengyuan Lv, Jingming Li, Li Yao, Xiaojuan Guo
Objective. Understanding the intricate relationship between structural connectivity (SC) and functional connectivity (FC) is pivotal for understanding the complexities of the human brain. To explore this relationship, the heat diffusion model (HDM) was utilized to predict FC from SC. However, previous studies using the HDM have typically predicted FC at a critical time scale in the heat kernel equation, overlooking the dynamic nature of the diffusion process and providing an incomplete representation of the predicted FC. Approach. In this study, we propose an alternative approach based on the HDM. First, we introduced a multiple-timescale fusion method to capture the dynamic features of the diffusion process. Additionally, to enhance the smoothness of the predicted FC values, we employed the Wavelet reconstruction method to maintain local consistency and remove noise. Moreover, to provide a more accurate representation of the relationship between SC and FC, we calculated the linear transformation between the smoothed FC and the empirical FC. Main results. We conducted extensive experiments in two independent datasets. By fusing different time scales in the diffusion process for predicting FC, the proposed method demonstrated higher predictive correlation compared with method considering only critical time points (Singlescale). Furthermore, compared with other existing methods, the proposed method achieved the highest predictive correlations of 0.6939 ± 0.0079 and 0.7302 ± 0.0117 on the two datasets respectively. We observed that the visual network at the network level and the parietal lobe at the lobe level exhibited the highest predictive correlations, indicating that the functional activity in these regions may be closely related to the direct diffusion of information between brain regions. Significance. The multiple-timescale fusion method proposed in this study provides insights into the dynamic aspects of the diffusion process, contributing to a deeper understanding of how brain structure gives rise to brain function.
目的。了解结构连通性（SC）和功能连通性（FC）之间错综复杂的关系对于理解人类大脑的复杂性至关重要。为了探索这种关系，研究人员利用热扩散模型（HDM）从 SC 预测 FC。然而，以往使用 HDM 的研究通常是在热核方程的临界时间尺度上预测 FC，从而忽略了扩散过程的动态性质，导致预测的 FC 不完整。方法。在本研究中，我们提出了一种基于 HDM 的替代方法。首先，我们引入了一种多时间尺度融合方法，以捕捉扩散过程的动态特征。此外，为了提高预测 FC 值的平滑度，我们采用了小波重建方法来保持局部一致性并去除噪声。此外，为了更准确地表示 SC 和 FC 之间的关系，我们计算了平滑 FC 和经验 FC 之间的线性变换。主要结果。我们在两个独立的数据集上进行了广泛的实验。通过融合扩散过程中的不同时间尺度来预测 FC，与只考虑关键时间点（单一尺度）的方法相比，所提出的方法表现出更高的预测相关性。此外，与其他现有方法相比，所提出的方法在两个数据集上分别达到了 0.6939 ± 0.0079 和 0.7302 ± 0.0117 的最高预测相关性。我们观察到，网络层面的视觉网络和顶叶层面的顶叶表现出最高的预测相关性，这表明这些区域的功能活动可能与脑区之间的直接信息扩散密切相关。意义重大。本研究提出的多时间尺度融合方法有助于深入了解扩散过程的动态方面，有助于加深对大脑结构如何产生大脑功能的理解。
{"title":"Predicting resting-state brain functional connectivity from the structural connectome using the heat diffusion model: a multiple-timescale fusion method","authors":"Zhengyuan Lv, Jingming Li, Li Yao, Xiaojuan Guo","doi":"10.1088/1741-2552/ad39a6","DOIUrl":"https://doi.org/10.1088/1741-2552/ad39a6","url":null,"abstract":"<italic toggle=\"yes\">Objective.</italic> Understanding the intricate relationship between structural connectivity (SC) and functional connectivity (FC) is pivotal for understanding the complexities of the human brain. To explore this relationship, the heat diffusion model (HDM) was utilized to predict FC from SC. However, previous studies using the HDM have typically predicted FC at a critical time scale in the heat kernel equation, overlooking the dynamic nature of the diffusion process and providing an incomplete representation of the predicted FC. <italic toggle=\"yes\">Approach.</italic> In this study, we propose an alternative approach based on the HDM. First, we introduced a multiple-timescale fusion method to capture the dynamic features of the diffusion process. Additionally, to enhance the smoothness of the predicted FC values, we employed the Wavelet reconstruction method to maintain local consistency and remove noise. Moreover, to provide a more accurate representation of the relationship between SC and FC, we calculated the linear transformation between the smoothed FC and the empirical FC. <italic toggle=\"yes\">Main results.</italic> We conducted extensive experiments in two independent datasets. By fusing different time scales in the diffusion process for predicting FC, the proposed method demonstrated higher predictive correlation compared with method considering only critical time points (Singlescale). Furthermore, compared with other existing methods, the proposed method achieved the highest predictive correlations of 0.6939 <inline-formula>\u0000<tex-math><?CDATA $ pm $?></tex-math>\u0000<mml:math overflow=\"scroll\"><mml:mrow><mml:mo>±</mml:mo></mml:mrow></mml:math>\u0000<inline-graphic xlink:href=\"jnead39a6ieqn1.gif\" xlink:type=\"simple\"></inline-graphic>\u0000</inline-formula> 0.0079 and 0.7302 <inline-formula>\u0000<tex-math><?CDATA $ pm $?></tex-math>\u0000<mml:math overflow=\"scroll\"><mml:mrow><mml:mo>±</mml:mo></mml:mrow></mml:math>\u0000<inline-graphic xlink:href=\"jnead39a6ieqn2.gif\" xlink:type=\"simple\"></inline-graphic>\u0000</inline-formula> 0.0117 on the two datasets respectively. We observed that the visual network at the network level and the parietal lobe at the lobe level exhibited the highest predictive correlations, indicating that the functional activity in these regions may be closely related to the direct diffusion of information between brain regions. <italic toggle=\"yes\">Significance.</italic> The multiple-timescale fusion method proposed in this study provides insights into the dynamic aspects of the diffusion process, contributing to a deeper understanding of how brain structure gives rise to brain function.","PeriodicalId":16753,"journal":{"name":"Journal of neural engineering","volume":null,"pages":null},"PeriodicalIF":4.0,"publicationDate":"2024-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140611139","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Objective. The extensive application of electroencephalography (EEG) in brain-computer interfaces (BCIs) can be attributed to its non-invasive nature and capability to offer high-resolution data. The acquisition of EEG signals is a straightforward process, but the datasets associated with these signals frequently exhibit data scarcity and require substantial resources for proper labeling. Furthermore, there is a significant limitation in the generalization performance of EEG models due to the substantial inter-individual variability observed in EEG signals. Approach. To address these issues, we propose a novel self-supervised contrastive learning framework for decoding motor imagery (MI) signals in cross-subject scenarios. Specifically, we design an encoder combining convolutional neural network and attention mechanism. In the contrastive learning training stage, the network undergoes training with the pretext task of data augmentation to minimize the distance between pairs of homologous transformations while simultaneously maximizing the distance between pairs of heterologous transformations. It enhances the amount of data utilized for training and improves the network’s ability to extract deep features from original signals without relying on the true labels of the data. Main results. To evaluate our framework’s efficacy, we conduct extensive experiments on three public MI datasets: BCI IV IIa, BCI IV IIb, and HGD datasets. The proposed method achieves cross-subject classification accuracies of 67.32%, 82.34%, and 81.13% on the three datasets, demonstrating superior performance compared to existing methods. Significance. Therefore, this method has great promise for improving the performance of cross-subject transfer learning in MI-based BCI systems.
目的。脑电图（EEG）在脑机接口（BCI）中的广泛应用归功于其非侵入性和提供高分辨率数据的能力。采集脑电信号是一个简单直接的过程，但与这些信号相关的数据集经常表现出数据稀缺性，需要大量资源才能进行正确标记。此外，由于在脑电信号中观察到的个体间差异很大，因此脑电图模型的泛化性能受到很大限制。方法。为了解决这些问题，我们提出了一种新颖的自监督对比学习框架，用于解码跨受试者场景中的运动图像（MI）信号。具体来说，我们设计了一种结合卷积神经网络和注意力机制的编码器。在对比学习训练阶段，网络以数据增强为借口进行训练，以最小化同源变换对之间的距离，同时最大化异源变换对之间的距离。这不仅增加了用于训练的数据量，还提高了网络从原始信号中提取深度特征的能力，而无需依赖数据的真实标签。主要结果为了评估我们框架的功效，我们在三个公开的人工智能数据集上进行了广泛的实验：BCI IV IIa、BCI IV IIb 和 HGD 数据集。所提出的方法在这三个数据集上的跨主体分类准确率分别达到了 67.32%、82.34% 和 81.13%，与现有方法相比表现出了卓越的性能。意义重大。因此，该方法在提高基于 MI 的 BCI 系统中的跨主体迁移学习性能方面大有可为。
{"title":"Self-supervised contrastive learning for EEG-based cross-subject motor imagery recognition","authors":"Wenjie Li, Haoyu Li, Xinlin Sun, Huicong Kang, Shan An, Guoxin Wang, Zhongke Gao","doi":"10.1088/1741-2552/ad3986","DOIUrl":"https://doi.org/10.1088/1741-2552/ad3986","url":null,"abstract":"<italic toggle=\"yes\">Objective</italic>. The extensive application of electroencephalography (EEG) in brain-computer interfaces (BCIs) can be attributed to its non-invasive nature and capability to offer high-resolution data. The acquisition of EEG signals is a straightforward process, but the datasets associated with these signals frequently exhibit data scarcity and require substantial resources for proper labeling. Furthermore, there is a significant limitation in the generalization performance of EEG models due to the substantial inter-individual variability observed in EEG signals. <italic toggle=\"yes\">Approach</italic>. To address these issues, we propose a novel self-supervised contrastive learning framework for decoding motor imagery (MI) signals in cross-subject scenarios. Specifically, we design an encoder combining convolutional neural network and attention mechanism. In the contrastive learning training stage, the network undergoes training with the pretext task of data augmentation to minimize the distance between pairs of homologous transformations while simultaneously maximizing the distance between pairs of heterologous transformations. It enhances the amount of data utilized for training and improves the network’s ability to extract deep features from original signals without relying on the true labels of the data. <italic toggle=\"yes\">Main results</italic>. To evaluate our framework’s efficacy, we conduct extensive experiments on three public MI datasets: BCI IV IIa, BCI IV IIb, and HGD datasets. The proposed method achieves cross-subject classification accuracies of 67.32<inline-formula>\u0000<tex-math><?CDATA $%$?></tex-math>\u0000<mml:math overflow=\"scroll\"><mml:mrow><mml:mi mathvariant=\"normal\">%</mml:mi></mml:mrow></mml:math>\u0000<inline-graphic xlink:href=\"jnead3986ieqn1.gif\" xlink:type=\"simple\"></inline-graphic>\u0000</inline-formula>, 82.34<inline-formula>\u0000<tex-math><?CDATA $%$?></tex-math>\u0000<mml:math overflow=\"scroll\"><mml:mrow><mml:mi mathvariant=\"normal\">%</mml:mi></mml:mrow></mml:math>\u0000<inline-graphic xlink:href=\"jnead3986ieqn2.gif\" xlink:type=\"simple\"></inline-graphic>\u0000</inline-formula>, and 81.13<inline-formula>\u0000<tex-math><?CDATA $%$?></tex-math>\u0000<mml:math overflow=\"scroll\"><mml:mrow><mml:mi mathvariant=\"normal\">%</mml:mi></mml:mrow></mml:math>\u0000<inline-graphic xlink:href=\"jnead3986ieqn3.gif\" xlink:type=\"simple\"></inline-graphic>\u0000</inline-formula> on the three datasets, demonstrating superior performance compared to existing methods. <italic toggle=\"yes\">Significance</italic>. Therefore, this method has great promise for improving the performance of cross-subject transfer learning in MI-based BCI systems.","PeriodicalId":16753,"journal":{"name":"Journal of neural engineering","volume":null,"pages":null},"PeriodicalIF":4.0,"publicationDate":"2024-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140611268","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Objective. Recent studies have shown that integrating inertial measurement unit (IMU) signals with surface electromyographic (sEMG) can greatly improve hand gesture recognition (HGR) performance in applications such as prosthetic control and rehabilitation training. However, current deep learning models for multimodal HGR encounter difficulties in invasive modal fusion, complex feature extraction from heterogeneous signals, and limited inter-subject model generalization. To address these challenges, this study aims to develop an end-to-end and inter-subject transferable model that utilizes non-invasively fused sEMG and acceleration (ACC) data. Approach. The proposed non-invasive modal fusion-transformer (NIMFT) model utilizes 1D-convolutional neural networks-based patch embedding for local information extraction and employs a multi-head cross-attention (MCA) mechanism to non-invasively integrate sEMG and ACC signals, stabilizing the variability induced by sEMG. The proposed architecture undergoes detailed ablation studies after hyperparameter tuning. Transfer learning is employed by fine-tuning a pre-trained model on new subject and a comparative analysis is performed between the fine-tuning and subject-specific model. Additionally, the performance of NIMFT is compared to state-of-the-art fusion models. Main results. The NIMFT model achieved recognition accuracies of 93.91%, 91.02%, and 95.56% on the three action sets in the Ninapro DB2 dataset. The proposed embedding method and MCA outperformed the traditional invasive modal fusion transformer by 2.01% (embedding) and 1.23% (fusion), respectively. In comparison to subject-specific models, the fine-tuning model exhibited the highest average accuracy improvement of 2.26%, achieving a final accuracy of 96.13%. Moreover, the NIMFT model demonstrated superiority in terms of accuracy, recall, precision, and F1-score compared to the latest modal fusion models with similar model scale. Significance. The NIMFT is a novel end-to-end HGR model, utilizes a non-invasive MCA mechanism to integrate long-range intermodal information effectively. Compared to recent modal fusion models, it demonstrates superior performance in inter-subject experiments and offers higher training efficiency and accuracy levels through transfer learning than subject-specific approaches.
{"title":"Transferable non-invasive modal fusion-transformer (NIMFT) for end-to-end hand gesture recognition","authors":"Tianxiang Xu, Kunkun Zhao, Yuxiang Hu, Liang Li, Wei Wang, Fulin Wang, Yuxuan Zhou, Jianqing Li","doi":"10.1088/1741-2552/ad39a5","DOIUrl":"https://doi.org/10.1088/1741-2552/ad39a5","url":null,"abstract":"<italic toggle=\"yes\">Objective.</italic> Recent studies have shown that integrating inertial measurement unit (IMU) signals with surface electromyographic (sEMG) can greatly improve hand gesture recognition (HGR) performance in applications such as prosthetic control and rehabilitation training. However, current deep learning models for multimodal HGR encounter difficulties in invasive modal fusion, complex feature extraction from heterogeneous signals, and limited inter-subject model generalization. To address these challenges, this study aims to develop an end-to-end and inter-subject transferable model that utilizes non-invasively fused sEMG and acceleration (ACC) data. <italic toggle=\"yes\">Approach.</italic> The proposed non-invasive modal fusion-transformer (NIMFT) model utilizes 1D-convolutional neural networks-based patch embedding for local information extraction and employs a multi-head cross-attention (MCA) mechanism to non-invasively integrate sEMG and ACC signals, stabilizing the variability induced by sEMG. The proposed architecture undergoes detailed ablation studies after hyperparameter tuning. Transfer learning is employed by fine-tuning a pre-trained model on new subject and a comparative analysis is performed between the fine-tuning and subject-specific model. Additionally, the performance of NIMFT is compared to state-of-the-art fusion models. <italic toggle=\"yes\">Main results.</italic> The NIMFT model achieved recognition accuracies of 93.91%, 91.02%, and 95.56% on the three action sets in the Ninapro DB2 dataset. The proposed embedding method and MCA outperformed the traditional invasive modal fusion transformer by 2.01% (embedding) and 1.23% (fusion), respectively. In comparison to subject-specific models, the fine-tuning model exhibited the highest average accuracy improvement of 2.26%, achieving a final accuracy of 96.13%. Moreover, the NIMFT model demonstrated superiority in terms of accuracy, recall, precision, and F1-score compared to the latest modal fusion models with similar model scale. <italic toggle=\"yes\">Significance.</italic> The NIMFT is a novel end-to-end HGR model, utilizes a non-invasive MCA mechanism to integrate long-range intermodal information effectively. Compared to recent modal fusion models, it demonstrates superior performance in inter-subject experiments and offers higher training efficiency and accuracy levels through transfer learning than subject-specific approaches.","PeriodicalId":16753,"journal":{"name":"Journal of neural engineering","volume":null,"pages":null},"PeriodicalIF":4.0,"publicationDate":"2024-04-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140611326","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-04-09DOI: 10.1088/1741-2552/ad38db
Nicole Ille
Objective. The extended infomax algorithm for independent component analysis (ICA) can separate sub- and super-Gaussian signals but converges slowly as it uses stochastic gradient optimization. In this paper, an improved extended infomax algorithm is presented that converges much faster. Approach. Accelerated convergence is achieved by replacing the natural gradient learning rule of extended infomax by a fully-multiplicative orthogonal-group based update scheme of the ICA unmixing matrix, leading to an orthogonal extended infomax algorithm (OgExtInf). The computational performance of OgExtInf was compared with original extended infomax and with two fast ICA algorithms: the popular FastICA and Picard, a preconditioned limited-memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) algorithm belonging to the family of quasi-Newton methods. Main results. OgExtInf converges much faster than original extended infomax. For small-size electroencephalogram (EEG) data segments, as used for example in online EEG processing, OgExtInf is also faster than FastICA and Picard. Significance. OgExtInf may be useful for fast and reliable ICA, e.g. in online systems for epileptic spike and seizure detection or brain-computer interfaces.
{"title":"Orthogonal extended infomax algorithm","authors":"Nicole Ille","doi":"10.1088/1741-2552/ad38db","DOIUrl":"https://doi.org/10.1088/1741-2552/ad38db","url":null,"abstract":"<italic toggle=\"yes\">Objective.</italic> The extended infomax algorithm for independent component analysis (ICA) can separate sub- and super-Gaussian signals but converges slowly as it uses stochastic gradient optimization. In this paper, an improved extended infomax algorithm is presented that converges much faster. <italic toggle=\"yes\">Approach.</italic> Accelerated convergence is achieved by replacing the natural gradient learning rule of extended infomax by a fully-multiplicative orthogonal-group based update scheme of the ICA unmixing matrix, leading to an orthogonal extended infomax algorithm (OgExtInf). The computational performance of OgExtInf was compared with original extended infomax and with two fast ICA algorithms: the popular FastICA and Picard, a preconditioned limited-memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) algorithm belonging to the family of quasi-Newton methods. <italic toggle=\"yes\">Main results.</italic> OgExtInf converges much faster than original extended infomax. For small-size electroencephalogram (EEG) data segments, as used for example in online EEG processing, OgExtInf is also faster than FastICA and Picard. <italic toggle=\"yes\">Significance.</italic> OgExtInf may be useful for fast and reliable ICA, e.g. in online systems for epileptic spike and seizure detection or brain-computer interfaces.","PeriodicalId":16753,"journal":{"name":"Journal of neural engineering","volume":null,"pages":null},"PeriodicalIF":4.0,"publicationDate":"2024-04-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140611269","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Objective. Artificial nerve scaffolds composed of polymers have attracted great attention as an alternative for autologous nerve grafts recently. Due to their poor bioactivity, satisfactory nerve repair could not be achieved. To solve this problem, we introduced extracellular matrix (ECM) to optimize the materials. Approach. In this study, the ECM extracted from porcine nerves was mixed with Poly(L-Lactide-co-ϵ-caprolactone) (PLCL), and the innovative PLCL/ECM nerve repair conduits were prepared by electrostatic spinning technology. The novel conduits were characterized by scanning electron microscopy (SEM), tensile properties, and suture retention strength test for micromorphology and mechanical strength. The biosafety and biocompatibility of PLCL/ECM nerve conduits were evaluated by cytotoxicity assay with Mouse fibroblast cells and cell adhesion assay with RSC 96 cells, and the effects of PLCL/ECM nerve conduits on the gene expression in Schwann cells was analyzed by real-time polymerase chain reaction (RT-PCR). Moreover, a 10 mm rat (Male Wistar rat) sciatic defect was bridged with a PLCL/ECM nerve conduit, and nerve regeneration was evaluated by walking track, mid-shank circumference, electrophysiology, and histomorphology analyses. Main results. The results showed that PLCL/ECM conduits have similar microstructure and mechanical strength compared with PLCL conduits. The cytotoxicity assay demonstrates better biosafety and biocompatibility of PLCL/ECM nerve conduits. And the cell adhesion assay further verifies that the addition of ECM is more beneficial to cell adhesion and proliferation. RT-PCR showed that the PLCL/ECM nerve conduit was more favorable to the gene expression of functional proteins of Schwann cells. The in vivo results indicated that PLCL/ECM nerve conduits possess excellent biocompatibility and exhibit a superior capacity to promote peripheral nerve repair. Significance. The addition of ECM significantly improved the biocompatibility and bioactivity of PLCL, while the PLCL/ECM nerve conduit gained the appropriate mechanical strength from PLCL, which has great potential for clinical repair of peripheral nerve injuries.
{"title":"Preparation of PLCL/ECM nerve conduits by electrostatic spinning technique and evaluation in vitro and in vivo","authors":"Yizhan Ma, Runze Zhang, Xiaoyan Mao, Xiaoming Li, Ting Li, Fang Liang, Jing He, Lili Wen, Weizuo Wang, Xiao Li, Yanhui Zhang, Honghao Yu, Binhan Lu, Tianhao Yu, Qiang Ao","doi":"10.1088/1741-2552/ad3851","DOIUrl":"https://doi.org/10.1088/1741-2552/ad3851","url":null,"abstract":"<italic toggle=\"yes\">Objective</italic>. Artificial nerve scaffolds composed of polymers have attracted great attention as an alternative for autologous nerve grafts recently. Due to their poor bioactivity, satisfactory nerve repair could not be achieved. To solve this problem, we introduced extracellular matrix (ECM) to optimize the materials. <italic toggle=\"yes\">Approach.</italic> In this study, the ECM extracted from porcine nerves was mixed with Poly(L-Lactide-co-<italic toggle=\"yes\">ϵ</italic>-caprolactone) (PLCL), and the innovative PLCL/ECM nerve repair conduits were prepared by electrostatic spinning technology. The novel conduits were characterized by scanning electron microscopy (SEM), tensile properties, and suture retention strength test for micromorphology and mechanical strength. The biosafety and biocompatibility of PLCL/ECM nerve conduits were evaluated by cytotoxicity assay with Mouse fibroblast cells and cell adhesion assay with RSC 96 cells, and the effects of PLCL/ECM nerve conduits on the gene expression in Schwann cells was analyzed by real-time polymerase chain reaction (RT-PCR). Moreover, a 10 mm rat (Male Wistar rat) sciatic defect was bridged with a PLCL/ECM nerve conduit, and nerve regeneration was evaluated by walking track, mid-shank circumference, electrophysiology, and histomorphology analyses. <italic toggle=\"yes\">Main results.</italic> The results showed that PLCL/ECM conduits have similar microstructure and mechanical strength compared with PLCL conduits. The cytotoxicity assay demonstrates better biosafety and biocompatibility of PLCL/ECM nerve conduits. And the cell adhesion assay further verifies that the addition of ECM is more beneficial to cell adhesion and proliferation. RT-PCR showed that the PLCL/ECM nerve conduit was more favorable to the gene expression of functional proteins of Schwann cells. The <italic toggle=\"yes\">in vivo</italic> results indicated that PLCL/ECM nerve conduits possess excellent biocompatibility and exhibit a superior capacity to promote peripheral nerve repair. <italic toggle=\"yes\">Significance.</italic> The addition of ECM significantly improved the biocompatibility and bioactivity of PLCL, while the PLCL/ECM nerve conduit gained the appropriate mechanical strength from PLCL, which has great potential for clinical repair of peripheral nerve injuries.","PeriodicalId":16753,"journal":{"name":"Journal of neural engineering","volume":null,"pages":null},"PeriodicalIF":4.0,"publicationDate":"2024-04-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140575769","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-08DOI: 10.1088/1741-2552/ad1385
Tao Xie, T. Foutz, M. Adamek, James R Swift, Cory S Inman, Joseph R Manns, E. Leuthardt, Jon T. Willie, Peter Brunner
Objective: Single-pulse electrical stimulation (SPES) has been widely used to probe effective connectivity. However, analysis of the neural response is often confounded by stimulation artifacts. We developed a novel matching pursuit-based artifact reconstruction and removal method (MPARRM) capable of removing artifacts from stimulation-artifact-affected electrophysiological signals. Approach: To validate MPARRM across a wide range of potential stimulation artifact types, we performed a bench-top experiment in which we suspended electrodes in a saline solution to generate 110 types of real-world stimulation artifacts. We then added the generated stimulation artifacts to ground truth signals (stereoelectroencephalography signals from 9 human subjects recorded during a receptive speech task), applied MPARRM to the combined signal, and compared the resultant denoised signal with the ground truth signal. We further applied MPARRM to artifact-affected neural signals recorded from the hippocampus while performing SPES on the ipsilateral basolateral amygdala in 9 human subjects. Results: MPARRM could remove stimulation artifacts without introducing spectral leakage or temporal spread. It accommodated variable stimulation parameters and recovered the early response to SPES within a wide range of frequency bands. Specifically, in the early response period (5 to 10 ms following stimulation onset), we found that the broadband gamma power (70-170 Hz) of the denoised signal was highly correlated with the ground truth signal (R=0.98±0.02, Pearson), and the broadband gamma activity of the denoised signal faithfully revealed the responses to the auditory stimuli within the ground truth signal with 94±1.47% sensitivity and 99±1.01% specificity. We further found that MPARRM could reveal the expected temporal progression of broadband gamma activity along the anterior-posterior axis of the hippocampus in response to the ipsilateral amygdala stimulation. Significance: MPARRM could faithfully remove SPES artifacts without confounding the electrophysiological signal components, especially during the early-response period. This method can facilitate the understanding of the neural response mechanisms of SPES.
{"title":"Single-pulse electrical stimulation artifact removal using the novel matching pursuit-based artifact reconstruction and removal method (MPARRM)","authors":"Tao Xie, T. Foutz, M. Adamek, James R Swift, Cory S Inman, Joseph R Manns, E. Leuthardt, Jon T. Willie, Peter Brunner","doi":"10.1088/1741-2552/ad1385","DOIUrl":"https://doi.org/10.1088/1741-2552/ad1385","url":null,"abstract":"\u0000 Objective: Single-pulse electrical stimulation (SPES) has been widely used to probe effective connectivity. However, analysis of the neural response is often confounded by stimulation artifacts. We developed a novel matching pursuit-based artifact reconstruction and removal method (MPARRM) capable of removing artifacts from stimulation-artifact-affected electrophysiological signals. Approach: To validate MPARRM across a wide range of potential stimulation artifact types, we performed a bench-top experiment in which we suspended electrodes in a saline solution to generate 110 types of real-world stimulation artifacts. We then added the generated stimulation artifacts to ground truth signals (stereoelectroencephalography signals from 9 human subjects recorded during a receptive speech task), applied MPARRM to the combined signal, and compared the resultant denoised signal with the ground truth signal. We further applied MPARRM to artifact-affected neural signals recorded from the hippocampus while performing SPES on the ipsilateral basolateral amygdala in 9 human subjects. Results: MPARRM could remove stimulation artifacts without introducing spectral leakage or temporal spread. It accommodated variable stimulation parameters and recovered the early response to SPES within a wide range of frequency bands. Specifically, in the early response period (5 to 10 ms following stimulation onset), we found that the broadband gamma power (70-170 Hz) of the denoised signal was highly correlated with the ground truth signal (R=0.98±0.02, Pearson), and the broadband gamma activity of the denoised signal faithfully revealed the responses to the auditory stimuli within the ground truth signal with 94±1.47% sensitivity and 99±1.01% specificity. We further found that MPARRM could reveal the expected temporal progression of broadband gamma activity along the anterior-posterior axis of the hippocampus in response to the ipsilateral amygdala stimulation. Significance: MPARRM could faithfully remove SPES artifacts without confounding the electrophysiological signal components, especially during the early-response period. This method can facilitate the understanding of the neural response mechanisms of SPES.","PeriodicalId":16753,"journal":{"name":"Journal of neural engineering","volume":null,"pages":null},"PeriodicalIF":4.0,"publicationDate":"2023-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138589782","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-01DOI: 10.1088/1741-2552/ad038d
Christian Y Song, M. Shanechi
Objective. Investigating neural population dynamics underlying behavior requires learning accurate models of the recorded spiking activity, which can be modeled with a Poisson observation distribution. Switching dynamical system models can offer both explanatory power and interpretability by piecing together successive regimes of simpler dynamics to capture more complex ones. However, in many cases, reliable regime labels are not available, thus demanding accurate unsupervised learning methods for Poisson observations. Existing learning methods, however, rely on inference of latent states in neural activity using the Laplace approximation, which may not capture the broader properties of densities and may lead to inaccurate learning. Thus, there is a need for new inference methods that can enable accurate model learning. Approach. To achieve accurate model learning, we derive a novel inference method based on deterministic sampling for Poisson observations called the Poisson Cubature Filter (PCF) and embed it in an unsupervised learning framework. This method takes a minimum mean squared error approach to estimation. Terms that are difficult to find analytically for Poisson observations are approximated in a novel way with deterministic sampling based on numerical integration and cubature rules. Main results. PCF enabled accurate unsupervised learning in both stationary and switching dynamical systems and largely outperformed prior Laplace approximation-based learning methods in both simulations and motor cortical spiking data recorded during a reaching task. These improvements were larger for smaller data sizes, showing that PCF-based learning was more data efficient and enabled more reliable regime identification. In experimental data and unsupervised with respect to behavior, PCF-based learning uncovered interpretable behavior-relevant regimes unlike prior learning methods. Significance. The developed unsupervised learning methods for switching dynamical systems can accurately uncover latent regimes and states in population spiking activity, with important applications in both basic neuroscience and neurotechnology.
{"title":"Unsupervised learning of stationary and switching dynamical system models from Poisson observations","authors":"Christian Y Song, M. Shanechi","doi":"10.1088/1741-2552/ad038d","DOIUrl":"https://doi.org/10.1088/1741-2552/ad038d","url":null,"abstract":"Objective. Investigating neural population dynamics underlying behavior requires learning accurate models of the recorded spiking activity, which can be modeled with a Poisson observation distribution. Switching dynamical system models can offer both explanatory power and interpretability by piecing together successive regimes of simpler dynamics to capture more complex ones. However, in many cases, reliable regime labels are not available, thus demanding accurate unsupervised learning methods for Poisson observations. Existing learning methods, however, rely on inference of latent states in neural activity using the Laplace approximation, which may not capture the broader properties of densities and may lead to inaccurate learning. Thus, there is a need for new inference methods that can enable accurate model learning. Approach. To achieve accurate model learning, we derive a novel inference method based on deterministic sampling for Poisson observations called the Poisson Cubature Filter (PCF) and embed it in an unsupervised learning framework. This method takes a minimum mean squared error approach to estimation. Terms that are difficult to find analytically for Poisson observations are approximated in a novel way with deterministic sampling based on numerical integration and cubature rules. Main results. PCF enabled accurate unsupervised learning in both stationary and switching dynamical systems and largely outperformed prior Laplace approximation-based learning methods in both simulations and motor cortical spiking data recorded during a reaching task. These improvements were larger for smaller data sizes, showing that PCF-based learning was more data efficient and enabled more reliable regime identification. In experimental data and unsupervised with respect to behavior, PCF-based learning uncovered interpretable behavior-relevant regimes unlike prior learning methods. Significance. The developed unsupervised learning methods for switching dynamical systems can accurately uncover latent regimes and states in population spiking activity, with important applications in both basic neuroscience and neurotechnology.","PeriodicalId":16753,"journal":{"name":"Journal of neural engineering","volume":null,"pages":null},"PeriodicalIF":4.0,"publicationDate":"2023-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138608182","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-27DOI: 10.1088/1741-2552/acf7f6
Xiaolin Xiao, Lijie Wang, Minpeng Xu, Kun Wang, Tzyy-Ping Jung, Dong Ming
Objective.Currently, steady-state visual evoked potentials (SSVEPs)-based brain-computer interfaces (BCIs) have achieved the highest interaction accuracy and speed among all BCI paradigms. However, its decoding efficacy depends deeply on the number of training samples, and the system performance would have a dramatic drop when the training dataset decreased to a small size. To date, no study has been reported to incorporate the unsupervised learning information from testing trails into the construction of supervised classification model, which is a potential way to mitigate the overfitting effect of limited samples.Approach.This study proposed a novel method for SSVEPs detection, i.e. cyclic shift trials (CSTs), which could combine unsupervised learning information from test trials and supervised learning information from train trials. Furthermore, since SSVEPs are time-locked and phase-locked to the onset of specific flashes, CST could also expand training samples on the basis of its regularity and periodicity. In order to verify the effectiveness of CST, we designed an online SSVEP-BCI system, and tested this system combined CST with two common classification algorithms, i.e. extended canonical correlation analysis and ensemble task-related component analysis.Main results.CST could significantly enhance the signal to noise ratios of SSVEPs and improve the performance of systems especially for the condition of few training samples and short stimulus time. The online information transfer rate could reach up to 236.19 bits min-1using 36 s calibration time of only one training sample for each category.Significance.The proposed CST method can take full advantages of supervised learning information from training samples and unsupervised learning information of testing samples. Furthermore, it is a data expansion technique, which can enhance the SSVEP characteristics and reduce dependence on sample size. Above all, CST is a promising method to improve the performance of SSVEP-based BCI without any additional experimental burden.
{"title":"A data expansion technique based on training and testing sample to boost the detection of SSVEPs for brain-computer interfaces.","authors":"Xiaolin Xiao, Lijie Wang, Minpeng Xu, Kun Wang, Tzyy-Ping Jung, Dong Ming","doi":"10.1088/1741-2552/acf7f6","DOIUrl":"10.1088/1741-2552/acf7f6","url":null,"abstract":"<p><p><i>Objective.</i>Currently, steady-state visual evoked potentials (SSVEPs)-based brain-computer interfaces (BCIs) have achieved the highest interaction accuracy and speed among all BCI paradigms. However, its decoding efficacy depends deeply on the number of training samples, and the system performance would have a dramatic drop when the training dataset decreased to a small size. To date, no study has been reported to incorporate the unsupervised learning information from testing trails into the construction of supervised classification model, which is a potential way to mitigate the overfitting effect of limited samples.<i>Approach.</i>This study proposed a novel method for SSVEPs detection, i.e. cyclic shift trials (CSTs), which could combine unsupervised learning information from test trials and supervised learning information from train trials. Furthermore, since SSVEPs are time-locked and phase-locked to the onset of specific flashes, CST could also expand training samples on the basis of its regularity and periodicity. In order to verify the effectiveness of CST, we designed an online SSVEP-BCI system, and tested this system combined CST with two common classification algorithms, i.e. extended canonical correlation analysis and ensemble task-related component analysis.<i>Main results.</i>CST could significantly enhance the signal to noise ratios of SSVEPs and improve the performance of systems especially for the condition of few training samples and short stimulus time. The online information transfer rate could reach up to 236.19 bits min<sup>-1</sup>using 36 s calibration time of only one training sample for each category.<i>Significance.</i>The proposed CST method can take full advantages of supervised learning information from training samples and unsupervised learning information of testing samples. Furthermore, it is a data expansion technique, which can enhance the SSVEP characteristics and reduce dependence on sample size. Above all, CST is a promising method to improve the performance of SSVEP-based BCI without any additional experimental burden.</p>","PeriodicalId":16753,"journal":{"name":"Journal of neural engineering","volume":null,"pages":null},"PeriodicalIF":4.0,"publicationDate":"2023-11-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10178276","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：