Pub Date : 2025-10-23DOI: 10.1109/THMS.2025.3616947
Zhongpan Zhu;Shuaijie Zhao;Mobing Cai;Cheng Wang;Aimin Du
Intelligent driving aims to handle dynamic driving tasks in complex environments, while driver behavior onboard is less focused. In contrast, an intelligent cockpit mainly focuses on interacting with a driver, with limited connection to the driving scenarios. Since the driver onboard could affect the driving strategy significantly and thus have nonnegligible safety implications on an autonomous vehicle, a cockpit-driving integration (CDI) is generally essential to take the driver’s behavior and intention into account when shaping the driving strategy. However, no comprehensive review of current existing CDI technologies is conducted despite the significant role of CDI in safe driving. Therefore, we are motivated to summarize the state-of-the-art of CDI methods and investigate the development trends of CDI. To this end, we identify thoroughly current applications of CDI for the perception and decision-making of autonomous vehicles and highlight critical issues that urgently need to be addressed. Additionally, we propose a lifelong learning framework based on evolvable neural networks as solutions for future CDI. Finally, challenges and future work are discussed. The work provides useful insights for developers regarding designing safe and human-centric autonomous vehicles.
{"title":"Automotive Cockpit-Driving Integration for Human-Centric Autonomous Driving: A Survey","authors":"Zhongpan Zhu;Shuaijie Zhao;Mobing Cai;Cheng Wang;Aimin Du","doi":"10.1109/THMS.2025.3616947","DOIUrl":"https://doi.org/10.1109/THMS.2025.3616947","url":null,"abstract":"Intelligent driving aims to handle dynamic driving tasks in complex environments, while driver behavior onboard is less focused. In contrast, an intelligent cockpit mainly focuses on interacting with a driver, with limited connection to the driving scenarios. Since the driver onboard could affect the driving strategy significantly and thus have nonnegligible safety implications on an autonomous vehicle, a cockpit-driving integration (CDI) is generally essential to take the driver’s behavior and intention into account when shaping the driving strategy. However, no comprehensive review of current existing CDI technologies is conducted despite the significant role of CDI in safe driving. Therefore, we are motivated to summarize the state-of-the-art of CDI methods and investigate the development trends of CDI. To this end, we identify thoroughly current applications of CDI for the perception and decision-making of autonomous vehicles and highlight critical issues that urgently need to be addressed. Additionally, we propose a lifelong learning framework based on evolvable neural networks as solutions for future CDI. Finally, challenges and future work are discussed. The work provides useful insights for developers regarding designing safe and human-centric autonomous vehicles.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"55 6","pages":"1016-1032"},"PeriodicalIF":4.4,"publicationDate":"2025-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145652146","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-10-17DOI: 10.1109/THMS.2025.3616313
Guobin Zhang;Keliang Li;Qiyuan Sun;Wenqi Wu;Shuai Li;Zhenzhong Liu
Traditional neurosurgical training modes face challenges including high costs, limited resources, lengthy learning curves, and difficulties in personalized training. In this article, we developed an immersive neurosurgical craniotomy virtual training system (NeuroSimulator) that integrates haptic feedback, enabling comprehensive surgical skill learning through an operator-control interface. Specifically, we constructed the comprehensive neurosurgical craniotomy surgical procedural (CNCSP) dataset to guide operators in repetitive learning and personalized training of relevant surgical skills. To address surgical site model rendering complexity, we proposed an algorithm that integrates vertex curvature and edge-length cost calculation factors (VC&ECL-QEM), resolving the incompatibility between surgical area image rendering quality and efficiency. For intracranial soft tissue haptic deformation, we developed a hybrid soft tissue haptic deformation (HBD) model that combines mass-spring and volumetric elements, addressing the collapse and distortion issues of traditional models and achieving more realistic soft tissue haptic deformation. Experimental results demonstrate that VC&ECL-QEM can simplify nonsurgical area feature preservation while maintaining surgical site detail features, reflecting the effectiveness of model simplification. The HBD model focuses on improving soft tissue deformation realism and shows high consistency with finite element model deformation effects. A total of 83 participants highly recognized NeuroSimulator’s system performance in terms of operational compliance, rendering real-time performance, and deformation realism, achieving effective improvements in skill proficiency metrics including operation time, ineffective operations, guidance requests, and operation scores. NeuroSimulator provides an innovative, efficient, and practical solution for neurosurgical training and is expected to play an increasingly important role in medical education and clinical skill enhancement.
{"title":"A Neurosurgical Craniotomy Training System Based on Haptic Virtual Reality Simulation","authors":"Guobin Zhang;Keliang Li;Qiyuan Sun;Wenqi Wu;Shuai Li;Zhenzhong Liu","doi":"10.1109/THMS.2025.3616313","DOIUrl":"https://doi.org/10.1109/THMS.2025.3616313","url":null,"abstract":"Traditional neurosurgical training modes face challenges including high costs, limited resources, lengthy learning curves, and difficulties in personalized training. In this article, we developed an immersive neurosurgical craniotomy virtual training system (NeuroSimulator) that integrates haptic feedback, enabling comprehensive surgical skill learning through an operator-control interface. Specifically, we constructed the comprehensive neurosurgical craniotomy surgical procedural (CNCSP) dataset to guide operators in repetitive learning and personalized training of relevant surgical skills. To address surgical site model rendering complexity, we proposed an algorithm that integrates vertex curvature and edge-length cost calculation factors (VC&ECL-QEM), resolving the incompatibility between surgical area image rendering quality and efficiency. For intracranial soft tissue haptic deformation, we developed a hybrid soft tissue haptic deformation (HBD) model that combines mass-spring and volumetric elements, addressing the collapse and distortion issues of traditional models and achieving more realistic soft tissue haptic deformation. Experimental results demonstrate that VC&ECL-QEM can simplify nonsurgical area feature preservation while maintaining surgical site detail features, reflecting the effectiveness of model simplification. The HBD model focuses on improving soft tissue deformation realism and shows high consistency with finite element model deformation effects. A total of 83 participants highly recognized NeuroSimulator’s system performance in terms of operational compliance, rendering real-time performance, and deformation realism, achieving effective improvements in skill proficiency metrics including operation time, ineffective operations, guidance requests, and operation scores. NeuroSimulator provides an innovative, efficient, and practical solution for neurosurgical training and is expected to play an increasingly important role in medical education and clinical skill enhancement.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"55 6","pages":"953-962"},"PeriodicalIF":4.4,"publicationDate":"2025-10-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145652176","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-10-10DOI: 10.1109/THMS.2025.3614336
{"title":"IEEE Systems, Man, and Cybernetics Society Information","authors":"","doi":"10.1109/THMS.2025.3614336","DOIUrl":"https://doi.org/10.1109/THMS.2025.3614336","url":null,"abstract":"","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"55 5","pages":"C3-C3"},"PeriodicalIF":4.4,"publicationDate":"2025-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=11200021","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145255898","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-10-10DOI: 10.1109/THMS.2025.3614338
{"title":"IEEE Transactions on Human-Machine Systems Information for Authors","authors":"","doi":"10.1109/THMS.2025.3614338","DOIUrl":"https://doi.org/10.1109/THMS.2025.3614338","url":null,"abstract":"","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"55 5","pages":"C4-C4"},"PeriodicalIF":4.4,"publicationDate":"2025-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=11200023","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145255896","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-10-08DOI: 10.1109/THMS.2025.3614364
Zekun Wu;Anna Maria Feit
Monitoring interfaces are crucial for dynamic, high-stakes tasks where effective user attention is essential. Visual highlights can guide attention effectively, but may also introduce unintended disruptions. To investigate this, we examined how visual highlights affect users’ gaze behavior in a drone monitoring task, focusing on when, how long, and how much attention they draw. We found that highlighted areas exhibit distinct temporal characteristics compared to nonhighlighted ones, quantified using normalized saliency (NS) metrics. We found that highlights elicited immediate responses, with NS peaking quickly, but this shift came at the cost of reduced search efforts elsewhere, potentially impacting situational awareness. To predict these dynamic changes and support interface design, we developed the Highlight-Informed Saliency Model, which provides granular predictions of NS over time. These predictions enable evaluations of highlight effectiveness and inform the optimal timing and deployment of highlights in future monitoring interface designs, particularly for time-sensitive tasks.
{"title":"Understanding and Predicting Temporal Visual Attention Influenced by Dynamic Highlights in Monitoring Task","authors":"Zekun Wu;Anna Maria Feit","doi":"10.1109/THMS.2025.3614364","DOIUrl":"https://doi.org/10.1109/THMS.2025.3614364","url":null,"abstract":"Monitoring interfaces are crucial for dynamic, high-stakes tasks where effective user attention is essential. Visual highlights can guide attention effectively, but may also introduce unintended disruptions. To investigate this, we examined how visual highlights affect users’ gaze behavior in a drone monitoring task, focusing on when, how long, and how much attention they draw. We found that highlighted areas exhibit distinct temporal characteristics compared to nonhighlighted ones, quantified using normalized saliency (NS) metrics. We found that highlights elicited immediate responses, with NS peaking quickly, but this shift came at the cost of reduced search efforts elsewhere, potentially impacting situational awareness. To predict these dynamic changes and support interface design, we developed the Highlight-Informed Saliency Model, which provides granular predictions of NS over time. These predictions enable evaluations of highlight effectiveness and inform the optimal timing and deployment of highlights in future monitoring interface designs, particularly for time-sensitive tasks.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"55 6","pages":"1053-1063"},"PeriodicalIF":4.4,"publicationDate":"2025-10-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145652163","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-10-08DOI: 10.1109/THMS.2025.3613773
Wei Wang;Qingzhong Li
The problem we focused on in this article is sensor-based generalized few-shot activity recognition. In this problem, each of the predefined activity classes (i.e., base classes) has substantial training instances, while each of the new activity classes (i.e., novel classes) just has a few training instances. Both the base and the novel classes need to be recognized. Currently, just a few works focus on this problem, and no formal statement of the problem is provided. In this article, we provide a formal definition of the problem, and propose a method to address it. In the proposed method, adopting the strategy of fine-tuning deep learning models, a deep learning model is first learned with the base-class training instances, and then fine-tuned with resampled training instances from both the base and the novel classes. We evaluate our method with three publicly available datasets on 1-shot, 5-shot, and 10-shot learning tasks. The results on the evaluation metric of harmonic mean of the average per-class accuracy for the base classes and that for the novel classes show that, our method could outperform state-of-the-art methods. In addition, the time and resource cost of our method is moderate.
{"title":"Deep Learning Model With Fine-Tuning for Generalized Few-Shot Activity Recognition","authors":"Wei Wang;Qingzhong Li","doi":"10.1109/THMS.2025.3613773","DOIUrl":"https://doi.org/10.1109/THMS.2025.3613773","url":null,"abstract":"The problem we focused on in this article is sensor-based generalized few-shot activity recognition. In this problem, each of the predefined activity classes (i.e., base classes) has substantial training instances, while each of the new activity classes (i.e., novel classes) just has a few training instances. Both the base and the novel classes need to be recognized. Currently, just a few works focus on this problem, and no formal statement of the problem is provided. In this article, we provide a formal definition of the problem, and propose a method to address it. In the proposed method, adopting the strategy of fine-tuning deep learning models, a deep learning model is first learned with the base-class training instances, and then fine-tuned with resampled training instances from both the base and the novel classes. We evaluate our method with three publicly available datasets on 1-shot, 5-shot, and 10-shot learning tasks. The results on the evaluation metric of harmonic mean of the average per-class accuracy for the base classes and that for the novel classes show that, our method could outperform state-of-the-art methods. In addition, the time and resource cost of our method is moderate.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"55 6","pages":"940-952"},"PeriodicalIF":4.4,"publicationDate":"2025-10-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145652155","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In skeleton-based gesture recognition tasks, existing approaches based on graph convolutional networks (GCNs) struggle to capture the synergistic actions of nonadjacent graph nodes and the information conveyed by their long-range dependencies. Combining spatial and temporal transformers is a promising solution to address the limitation, inspired by the advantage of transformer in assessing nonadjacent long-range dependencies, but there lacks an effective strategy to integrate the spatial and temporal information extracted by these transformers. Therefore, this article proposes the spatial-temporal alternating graph convolution transformer (ST-GCN-AltFormer), which connects the spatial-temporal graph convolutional network (ST-GCN) with the spatial-temporal alternating transformer (AltFormer) architecture. In the AltFormer architecture, the spatial-temporal transformer branch employs a spatial transformer to capture information from specific frames, and uses a temporal transformer to analyze its evolution over the entire temporal range. Meanwhile, the temporal-spatial transformer branch extracts temporal information from specific nodes using a temporal transformer, and integrates it with a spatial transformer. The fusion enhances accurate spatial-temporal information extraction. Our method achieves superior performance compared to state-of-the-art methods, achieving accuracies of 97.5%, 95.8%, 94.3%, 92.8%, and 98.31% on the large-scale 3D hand gesture recognition (SHREC’17 Track), Dynamic Hand Gesture 14-28 (DHG-14/28), and leap motion dynamic hand gesture (LMDHG) dynamic gesture datasets, respectively.
{"title":"ST-GCN-AltFormer: Gesture Recognition With Spatial-Temporal Alternating Transformer","authors":"Qing Pan;Jintao Zhu;Lingwei Zhang;Gangmin Ning;Luping Fang","doi":"10.1109/THMS.2025.3607961","DOIUrl":"https://doi.org/10.1109/THMS.2025.3607961","url":null,"abstract":"In skeleton-based gesture recognition tasks, existing approaches based on graph convolutional networks (GCNs) struggle to capture the synergistic actions of nonadjacent graph nodes and the information conveyed by their long-range dependencies. Combining spatial and temporal transformers is a promising solution to address the limitation, inspired by the advantage of transformer in assessing nonadjacent long-range dependencies, but there lacks an effective strategy to integrate the spatial and temporal information extracted by these transformers. Therefore, this article proposes the spatial-temporal alternating graph convolution transformer (ST-GCN-AltFormer), which connects the spatial-temporal graph convolutional network (ST-GCN) with the spatial-temporal alternating transformer (AltFormer) architecture. In the AltFormer architecture, the spatial-temporal transformer branch employs a spatial transformer to capture information from specific frames, and uses a temporal transformer to analyze its evolution over the entire temporal range. Meanwhile, the temporal-spatial transformer branch extracts temporal information from specific nodes using a temporal transformer, and integrates it with a spatial transformer. The fusion enhances accurate spatial-temporal information extraction. Our method achieves superior performance compared to state-of-the-art methods, achieving accuracies of 97.5%, 95.8%, 94.3%, 92.8%, and 98.31% on the large-scale 3D hand gesture recognition (SHREC’17 Track), Dynamic Hand Gesture 14-28 (DHG-14/28), and leap motion dynamic hand gesture (LMDHG) dynamic gesture datasets, respectively.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"55 6","pages":"963-972"},"PeriodicalIF":4.4,"publicationDate":"2025-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145652188","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-10-01Epub Date: 2025-08-18DOI: 10.1109/thms.2025.3592791
Melih Turkseven, Trudi Di Qi, Ganesh Sankaranarayanan, Suvranu De
Cricothyroidotomy (CCT) is a critical, life-saving procedure requiring the identification of key neck landmarks through palpation. Interactive virtual simulation offers a promising, cost-effective approach to CCT training with high visual realism. However, developing the palpation skills necessary for CCT requires a haptic interface with tactile sensitivity comparable to human fingers. Such interfaces are often represented by plastic partial mannequins, which require further adaptation to integrate into virtual environments. This study introduces an instrumented physical palpation interface for CCT, integrated into a virtual surgical simulator, and tested on ten surgeons who practiced the procedure over a training period. Data on haptic interactions collected during the training was analyzed to evaluate participants' palpation skills and explore their force modulation strategies about landmark identification scores. Our findings suggest that trainees become more precise in their exploration over time, apply greater normal forces around target areas. Initial landmark identification performance influences adjustments in the overall applied pressure.
{"title":"Palpation Characteristics of an Instrumented Virtual Cricothyroidotomy Simulator.","authors":"Melih Turkseven, Trudi Di Qi, Ganesh Sankaranarayanan, Suvranu De","doi":"10.1109/thms.2025.3592791","DOIUrl":"10.1109/thms.2025.3592791","url":null,"abstract":"<p><p>Cricothyroidotomy (CCT) is a critical, life-saving procedure requiring the identification of key neck landmarks through palpation. Interactive virtual simulation offers a promising, cost-effective approach to CCT training with high visual realism. However, developing the palpation skills necessary for CCT requires a haptic interface with tactile sensitivity comparable to human fingers. Such interfaces are often represented by plastic partial mannequins, which require further adaptation to integrate into virtual environments. This study introduces an instrumented physical palpation interface for CCT, integrated into a virtual surgical simulator, and tested on ten surgeons who practiced the procedure over a training period. Data on haptic interactions collected during the training was analyzed to evaluate participants' palpation skills and explore their force modulation strategies about landmark identification scores. Our findings suggest that trainees become more precise in their exploration over time, apply greater normal forces around target areas. Initial landmark identification performance influences adjustments in the overall applied pressure.</p>","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"55 5","pages":"876-885"},"PeriodicalIF":4.4,"publicationDate":"2025-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12617396/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145543563","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-09-18DOI: 10.1109/THMS.2025.3605011
Mojtaba Esfandiari;Peter Gehlbach;Russell H. Taylor;Iulian I. Iordachita
Performing retinal vein cannulation (RVC) as a potential treatment for retinal vein occlusion without the assistance of a surgical robotic system is very challenging to do safely. The main limitation is the physiological hand tremor of surgeons. Robot-assisted eye surgery technology may resolve the problems of hand tremors and fatigue and improve the safety and precision of RVC. The steady-hand eye robot (SHER) is an admittance-based robotic system that can filter out hand tremors and enables ophthalmologists to manipulate a surgical instrument inside the eye cooperatively. However, the admittance-based cooperative control mode does not safely minimize the contact force between the surgical instrument and the sclera to prevent tissue damage. In addition, features such as haptic feedback or hand motion scaling, which can improve the safety and precision of surgery, require a teleoperation control framework. This work presents, for the first time in the field of robot-assisted retinal microsurgery research, a registration-free bimanual adaptive teleoperation (BMAT) control framework using SHER 2.0 and SHER 2.1 robotic systems. Both SHERs are integrated with an adaptive force control algorithm that dynamically and automatically minimizes the tool–sclera interaction forces, enforcing them within a safe limit. The scleral forces are measured using two fiber Bragg grating-based force-sensing tools. The performance of the proposed BMAT control framework is evaluated by comparison with a bimanual adaptive cooperative framework in a vessel-following experiment conducted under a surgical microscope. Experimental results demonstrate the effectiveness of the BMAT control framework in performing a safe bimanual telemanipulation of the eye without overstretching it, even in the absence of registration between the two robots.
{"title":"Bimanual Manipulation of Steady-Hand Eye Robots With Adaptive Sclera Force Control: Cooperative Versus Teleoperation Strategies","authors":"Mojtaba Esfandiari;Peter Gehlbach;Russell H. Taylor;Iulian I. Iordachita","doi":"10.1109/THMS.2025.3605011","DOIUrl":"https://doi.org/10.1109/THMS.2025.3605011","url":null,"abstract":"Performing retinal vein cannulation (RVC) as a potential treatment for retinal vein occlusion without the assistance of a surgical robotic system is very challenging to do safely. The main limitation is the physiological hand tremor of surgeons. Robot-assisted eye surgery technology may resolve the problems of hand tremors and fatigue and improve the safety and precision of RVC. The steady-hand eye robot (SHER) is an admittance-based robotic system that can filter out hand tremors and enables ophthalmologists to manipulate a surgical instrument inside the eye cooperatively. However, the admittance-based cooperative control mode does not safely minimize the contact force between the surgical instrument and the sclera to prevent tissue damage. In addition, features such as haptic feedback or hand motion scaling, which can improve the safety and precision of surgery, require a teleoperation control framework. This work presents, for the first time in the field of robot-assisted retinal microsurgery research, a registration-free bimanual adaptive teleoperation (BMAT) control framework using SHER 2.0 and SHER 2.1 robotic systems. Both SHERs are integrated with an adaptive force control algorithm that dynamically and automatically minimizes the tool–sclera interaction forces, enforcing them within a safe limit. The scleral forces are measured using two fiber Bragg grating-based force-sensing tools. The performance of the proposed BMAT control framework is evaluated by comparison with a bimanual adaptive cooperative framework in a vessel-following experiment conducted under a surgical microscope. Experimental results demonstrate the effectiveness of the BMAT control framework in performing a safe bimanual telemanipulation of the eye without overstretching it, even in the absence of registration between the two robots.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"55 6","pages":"909-919"},"PeriodicalIF":4.4,"publicationDate":"2025-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145652177","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-09-12DOI: 10.1109/THMS.2025.3603548
Ericka R. da Silva Serafini;Cristian D. Guerrero-Mendez;Douglas M. Dunga;Teodiano F. Bastos-Filho;Anibal Cotrina Atencio;André F. O. de Azevedo Dantas;Caroline C. do Espírito Santo;Denis Delisle-Rodriguez
Robotic interventions combining neurofeedback (NFB) and motor imagery (MI) are emerging strategies to promote cortical reorganization and functional training in individuals with complete spinal cord injury (SCI). This study proposes an electroencephalogram-based NFB approach for MI training, designed to teach the MI-related brain rhythmics modulation in Lokomat. For the purposes of this study, NFB is defined as a visual feedback training scheme. The proposed system introduces a formulation to minimize the default cortical effects that Lokomat produces on the individual’s activity during passive walking. Two individuals with complete SCI tested the proposed NFB system, in order to relearn the modulation of Mu ($mu$ : 8–12 Hz) and Beta ($beta$ : 13–30 Hz) rhythms over Cz, while receiving gait training with full weight support across 12 sessions. Each session consisted of the following three stages: 1) 2 min walking without MI (baseline); 2) 5 min walking with MI and True NFB; and 3) 5 min walking with MI and Sham NFB. The latter two stages were randomized session-by-session. The findings suggest that the proposed NFB approach may promote cortical reorganization and support the restoration of sensorimotor functions. Significant differences were observed between cortical patterns during True NFB and Sham NFB, particularly in the last intervention sessions. These results confirm the positive impact of the NFB system on gait motor training by enabling individuals with complete SCI to learn how to modulate their motor rhythms in specific cortical areas.
{"title":"EEG Neurofeedback-Based Gait Motor Imagery Training in Lokomat Enhances Motor Rhythms in Complete Spinal Cord Injury","authors":"Ericka R. da Silva Serafini;Cristian D. Guerrero-Mendez;Douglas M. Dunga;Teodiano F. Bastos-Filho;Anibal Cotrina Atencio;André F. O. de Azevedo Dantas;Caroline C. do Espírito Santo;Denis Delisle-Rodriguez","doi":"10.1109/THMS.2025.3603548","DOIUrl":"https://doi.org/10.1109/THMS.2025.3603548","url":null,"abstract":"Robotic interventions combining neurofeedback (NFB) and motor imagery (MI) are emerging strategies to promote cortical reorganization and functional training in individuals with complete spinal cord injury (SCI). This study proposes an electroencephalogram-based NFB approach for MI training, designed to teach the MI-related brain rhythmics modulation in Lokomat. For the purposes of this study, NFB is defined as a visual feedback training scheme. The proposed system introduces a formulation to minimize the default cortical effects that Lokomat produces on the individual’s activity during passive walking. Two individuals with complete SCI tested the proposed NFB system, in order to relearn the modulation of Mu (<inline-formula><tex-math>$mu$</tex-math></inline-formula> : 8–12 Hz) and Beta (<inline-formula><tex-math>$beta$</tex-math></inline-formula> : 13–30 Hz) rhythms over Cz, while receiving gait training with full weight support across 12 sessions. Each session consisted of the following three stages: 1) 2 min walking without MI (baseline); 2) 5 min walking with MI and True NFB; and 3) 5 min walking with MI and Sham NFB. The latter two stages were randomized session-by-session. The findings suggest that the proposed NFB approach may promote cortical reorganization and support the restoration of sensorimotor functions. Significant differences were observed between cortical patterns during True NFB and Sham NFB, particularly in the last intervention sessions. These results confirm the positive impact of the NFB system on gait motor training by enabling individuals with complete SCI to learn how to modulate their motor rhythms in specific cortical areas.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"55 6","pages":"983-992"},"PeriodicalIF":4.4,"publicationDate":"2025-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=11163517","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145652152","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：