Predicting workers' body postures is crucial for effective ergonomic interventions to reduce musculoskeletal disorders (MSDs). In this study, we employ a novel generative approach to predict human postures during manual material handling tasks. Specifically, we implement two distinct network architectures, U-Net and multilayer perceptron (MLP), to build the diffusion model. The model training and testing utilizes a dataset featuring 35 full-body anatomical landmarks collected from 25 participants engaged in a variety of lifting tasks. In addition, we compare our models with two conventional generative networks (conditional generative adversarial network and conditional variational autoencoder) for comprehensive analysis. Our results show that the U-Net model performs well in predicting posture similarity [root-mean-square error (RMSE) of key-point coordinates = 5.86 cm; and RMSE of joint angle coordinates = 13.67 $^{circ }$ ], while the MLP model leads to higher posture variability (e.g., standard deviation of joint angles = 4.49 $^{circ }$ /4.18 $^{circ }$ for upper arm flexion/extension joints). Moreover, both generative models demonstrate reasonable prediction validity (RMSE of segment lengths are within 4.83 cm). Overall, our proposed diffusion models demonstrate good similarity and validity in predicting lifting postures, while also providing insights into the inherent variability of constrained lifting postures. This novel use of diffusion models shows potential for tailored posture prediction in common occupational environments, representing an advancement in motion synthesis and contributing to workplace design and MSD risk mitigation.
{"title":"Predicting Human Postures for Manual Material Handling Tasks Using a Conditional Diffusion Model","authors":"Liwei Qing;Bingyi Su;Sehee Jung;Lu Lu;Hanwen Wang;Xu Xu","doi":"10.1109/THMS.2024.3472548","DOIUrl":"https://doi.org/10.1109/THMS.2024.3472548","url":null,"abstract":"Predicting workers' body postures is crucial for effective ergonomic interventions to reduce musculoskeletal disorders (MSDs). In this study, we employ a novel generative approach to predict human postures during manual material handling tasks. Specifically, we implement two distinct network architectures, U-Net and multilayer perceptron (MLP), to build the diffusion model. The model training and testing utilizes a dataset featuring 35 full-body anatomical landmarks collected from 25 participants engaged in a variety of lifting tasks. In addition, we compare our models with two conventional generative networks (conditional generative adversarial network and conditional variational autoencoder) for comprehensive analysis. Our results show that the U-Net model performs well in predicting posture similarity [root-mean-square error (RMSE) of key-point coordinates = 5.86 cm; and RMSE of joint angle coordinates = 13.67\u0000<inline-formula><tex-math>$^{circ }$</tex-math></inline-formula>\u0000], while the MLP model leads to higher posture variability (e.g., standard deviation of joint angles = 4.49\u0000<inline-formula><tex-math>$^{circ }$</tex-math></inline-formula>\u0000/4.18\u0000<inline-formula><tex-math>$^{circ }$</tex-math></inline-formula>\u0000 for upper arm flexion/extension joints). Moreover, both generative models demonstrate reasonable prediction validity (RMSE of segment lengths are within 4.83 cm). Overall, our proposed diffusion models demonstrate good similarity and validity in predicting lifting postures, while also providing insights into the inherent variability of constrained lifting postures. This novel use of diffusion models shows potential for tailored posture prediction in common occupational environments, representing an advancement in motion synthesis and contributing to workplace design and MSD risk mitigation.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 6","pages":"723-732"},"PeriodicalIF":3.5,"publicationDate":"2024-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142691718","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-18DOI: 10.1109/THMS.2024.3467370
Kok-Lim Alvin Yau;Yasir Saleem;Yung-Wey Chong;Xiumei Fan;Jer Min Eyu;David Chieng
Augmented intelligence (AuI) is a concept that combines human intelligence (HI) and artificial intelligence (AI) to leverage their respective strengths. While AI typically aims to replace humans, AuI integrates humans into machines, recognizing their irreplaceable role. Meanwhile, human-in-the-loop reinforcement learning (HITL-RL) is a semisupervised algorithm that integrates humans into the traditional reinforcement learning (RL) algorithm, enabling autonomous agents to gather inputs from both humans and environments, learn, and select optimal actions across various environments. Both AuI and HITL-RL are still in their infancy. Based on AuI, we propose and investigate three separate concept designs for HITL-RL: HI-AI , AI-HI , and parallel-HI-and-AI approaches, each differing in the order of HI and AI involvement in decision making. The literature on AuI and HITL-RL offers insights into integrating HI into existing concept designs. A preliminary study in an Atari game offers insights for future research directions. Simulation results show that human involvement maintains RL convergence and improves system stability, while achieving approximately similar average scores to traditional $Q$ -learning in the game. Future research directions are proposed to encourage further investigation in this area.
{"title":"The Augmented Intelligence Perspective on Human-in-the-Loop Reinforcement Learning: Review, Concept Designs, and Future Directions","authors":"Kok-Lim Alvin Yau;Yasir Saleem;Yung-Wey Chong;Xiumei Fan;Jer Min Eyu;David Chieng","doi":"10.1109/THMS.2024.3467370","DOIUrl":"https://doi.org/10.1109/THMS.2024.3467370","url":null,"abstract":"Augmented intelligence (AuI) is a concept that combines human intelligence (HI) and artificial intelligence (AI) to leverage their respective strengths. While AI typically aims to replace humans, AuI integrates humans into machines, recognizing their irreplaceable role. Meanwhile, human-in-the-loop reinforcement learning (HITL-RL) is a semisupervised algorithm that integrates humans into the traditional reinforcement learning (RL) algorithm, enabling autonomous agents to gather inputs from both humans and environments, learn, and select optimal actions across various environments. Both AuI and HITL-RL are still in their infancy. Based on AuI, we propose and investigate three separate concept designs for HITL-RL: \u0000<italic>HI-AI</i>\u0000, \u0000<italic>AI-HI</i>\u0000, and \u0000<italic>parallel-HI-and-AI</i>\u0000 approaches, each differing in the order of HI and AI involvement in decision making. The literature on AuI and HITL-RL offers insights into integrating HI into existing concept designs. A preliminary study in an Atari game offers insights for future research directions. Simulation results show that human involvement maintains RL convergence and improves system stability, while achieving approximately similar average scores to traditional \u0000<inline-formula><tex-math>$Q$</tex-math></inline-formula>\u0000-learning in the game. Future research directions are proposed to encourage further investigation in this area.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 6","pages":"762-777"},"PeriodicalIF":3.5,"publicationDate":"2024-10-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142691214","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-17DOI: 10.1109/THMS.2024.3467334
Mengyuan Liu;Hong Liu;Tianyu Guo
Considering the instance-level discriminative ability, contrastive learning methods, including MoCo and SimCLR, have been adapted from the original image representation learning task to solve the self-supervised skeleton-based action recognition task. These methods usually use multiple data streams (i.e., joint, motion, and bone) for ensemble learning, meanwhile, how to construct a discriminative feature space within a single stream and effectively aggregate the information from multiple streams remains an open problem. To this end, this article first applies a new contrastive learning method called bootstrap your own latent (BYOL) to learn from skeleton data, and then formulate SkeletonBYOL as a simple yet effective baseline for self-supervised skeleton-based action recognition. Inspired by SkeletonBYOL, this article further presents a cross-model and cross-stream (CMCS) framework. This framework combines cross-model adversarial learning (CMAL) and cross-stream collaborative learning (CSCL). Specifically, CMAL learns single-stream representation by cross-model adversarial loss to obtain more discriminative features. To aggregate and interact with multistream information, CSCL is designed by generating similarity pseudolabel of ensemble learning as supervision and guiding feature generation for individual streams. Extensive experiments on three datasets verify the complementary properties between CMAL and CSCL and also verify that the proposed method can achieve better results than state-of-the-art methods using various evaluation protocols.
{"title":"Cross-Model Cross-Stream Learning for Self-Supervised Human Action Recognition","authors":"Mengyuan Liu;Hong Liu;Tianyu Guo","doi":"10.1109/THMS.2024.3467334","DOIUrl":"https://doi.org/10.1109/THMS.2024.3467334","url":null,"abstract":"Considering the instance-level discriminative ability, contrastive learning methods, including MoCo and SimCLR, have been adapted from the original image representation learning task to solve the self-supervised skeleton-based action recognition task. These methods usually use multiple data streams (i.e., joint, motion, and bone) for ensemble learning, meanwhile, how to construct a discriminative feature space within a single stream and effectively aggregate the information from multiple streams remains an open problem. To this end, this article first applies a new contrastive learning method called bootstrap your own latent (BYOL) to learn from skeleton data, and then formulate SkeletonBYOL as a simple yet effective baseline for self-supervised skeleton-based action recognition. Inspired by SkeletonBYOL, this article further presents a cross-model and cross-stream (CMCS) framework. This framework combines cross-model adversarial learning (CMAL) and cross-stream collaborative learning (CSCL). Specifically, CMAL learns single-stream representation by cross-model adversarial loss to obtain more discriminative features. To aggregate and interact with multistream information, CSCL is designed by generating similarity pseudolabel of ensemble learning as supervision and guiding feature generation for individual streams. Extensive experiments on three datasets verify the complementary properties between CMAL and CSCL and also verify that the proposed method can achieve better results than state-of-the-art methods using various evaluation protocols.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 6","pages":"743-752"},"PeriodicalIF":3.5,"publicationDate":"2024-10-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142691215","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-15DOI: 10.1109/THMS.2024.3468019
Nadia Cattari;Fabrizio Cutolo;Vincenzo Ferrari
For an optical see-through (OST) augmented reality (AR) head-mounted display (HMD) to assist in performing high-precision activities in the peripersonal space, a fundamental requirement is the correct spatial registration between the virtual information and the real environment. This registration can be achieved through a calibration procedure involving the parameterization of the virtual rendering camera via an eye-replacement camera that observes a calibration pattern rendered onto the OST display. In a previous feasibility study, we demonstrated and proved, with the same eye-replacement camera used for the calibration, that, in the case of an OST display with a focal plane close to the user's working distance, there is no need for prior-to-use viewpoint-specific calibration refinements obtained through eye-tracking cameras or additional alignment-based calibration steps. The viewpoint parallax-related AR registration error is indeed submillimetric within a reasonable range of depths around the display focal plane. This article confirms, through a user study based on a monocular virtual-to-real alignment task, that this finding is accurate and usable. In addition, we found that by performing the alignment-free calibration procedure via a high-resolution camera, the AR registration accuracy is substantially improved compared with that of other state-of-the-art approaches, with an error lower than 1mm over a notable range of distances. These results demonstrate the safe usability of OST HMDs for high-precision task guidance in the peripersonal space.
{"title":"Optical See-Through Head-Mounted Display With Mitigated Parallax-Related Registration Errors: A User Study Validation","authors":"Nadia Cattari;Fabrizio Cutolo;Vincenzo Ferrari","doi":"10.1109/THMS.2024.3468019","DOIUrl":"https://doi.org/10.1109/THMS.2024.3468019","url":null,"abstract":"For an optical see-through (OST) augmented reality (AR) head-mounted display (HMD) to assist in performing high-precision activities in the peripersonal space, a fundamental requirement is the correct spatial registration between the virtual information and the real environment. This registration can be achieved through a calibration procedure involving the parameterization of the virtual rendering camera via an eye-replacement camera that observes a calibration pattern rendered onto the OST display. In a previous feasibility study, we demonstrated and proved, with the same eye-replacement camera used for the calibration, that, in the case of an OST display with a focal plane close to the user's working distance, there is no need for prior-to-use viewpoint-specific calibration refinements obtained through eye-tracking cameras or additional alignment-based calibration steps. The viewpoint parallax-related AR registration error is indeed submillimetric within a reasonable range of depths around the display focal plane. This article confirms, through a user study based on a monocular virtual-to-real alignment task, that this finding is accurate and usable. In addition, we found that by performing the alignment-free calibration procedure via a high-resolution camera, the AR registration accuracy is substantially improved compared with that of other state-of-the-art approaches, with an error lower than 1mm over a notable range of distances. These results demonstrate the safe usability of OST HMDs for high-precision task guidance in the peripersonal space.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 6","pages":"668-677"},"PeriodicalIF":3.5,"publicationDate":"2024-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10718696","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142691688","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-09DOI: 10.1109/THMS.2024.3439625
Long Chen;Wei Zhang;Yanqing Song;Jianguo Chen
This article conducts a thorough exploration of the implications of machine learning (ML) in conjunction with human–machine systems within the military domain. It scrutinizes the strategic development efforts of ML by pertinent institutions, particularly in the context of military applications and the domain of advanced persistent threats. Prominent nations have delineated a technical trajectory for the integration of ML into their military frameworks. To bolster the structure and efficacy of their various military branches and units, there has been a concentrated deployment of numerous ML research endeavors. These initiatives encompass the study of sophisticated ML algorithms and the acceleration of artificial intelligence technology adaptation for intelligence processing, autonomous platforms, command and control infrastructures, and weapons systems. Forces across the globe are actively embedding ML technologies into a range of platforms-terrestrial, naval, aerial, space-faring, and cybernetic. This integration spans weaponry, networks, cognitive operations, and additional systems. Furthermore, this article reviews the incorporation within the sphere of military human–machine interaction in the Russia–Ukraine conflict. In this war, cyber human–machine interaction has become a pivotal arena of contention between Russia and Ukraine, with key levers that influence the conflict's course. In addition, the article examines the adoption of ML in prospective military functions such as, operations, intelligence gathering, networking, logistics, identification protocols, healthcare, data analysis trends, and other critical areas marked by current developments and trajectories. It also proffers a series of recommendations for the future integration of ML to inform strategic direction and research.
本文深入探讨了机器学习（ML）与人机系统在军事领域的结合所产生的影响。文章仔细研究了相关机构在 ML 战略发展方面所做的努力，特别是在军事应用和高级持久性威胁领域。著名国家已为将 ML 纳入其军事框架划定了技术轨迹。为了加强各军事部门和单位的结构和效率，各国集中部署了大量的 ML 研究工作。这些举措包括研究复杂的 ML 算法，加快人工智能技术在情报处理、自主平台、指挥和控制基础设施以及武器系统中的应用。全球各地的部队都在积极将 ML 技术嵌入各种平台--陆地平台、海军平台、航空平台、航天平台和控制论平台。这种集成涵盖武器装备、网络、认知操作和其他系统。此外，本文还回顾了俄乌冲突中军事人机互动领域的整合情况。在这场战争中，网络人机交互已成为俄罗斯和乌克兰之间争夺的关键领域，并成为影响冲突进程的关键杠杆。此外，文章还探讨了在未来军事功能中采用 ML 的情况，如作战、情报收集、网络、后勤、身份识别协议、医疗保健、数据分析趋势以及其他以当前发展和轨迹为标志的关键领域。文章还就未来如何整合 ML 提出了一系列建议，为战略方向和研究提供参考。
{"title":"Machine Learning for Human–Machine Systems With Advanced Persistent Threats","authors":"Long Chen;Wei Zhang;Yanqing Song;Jianguo Chen","doi":"10.1109/THMS.2024.3439625","DOIUrl":"https://doi.org/10.1109/THMS.2024.3439625","url":null,"abstract":"This article conducts a thorough exploration of the implications of machine learning (ML) in conjunction with human–machine systems within the military domain. It scrutinizes the strategic development efforts of ML by pertinent institutions, particularly in the context of military applications and the domain of advanced persistent threats. Prominent nations have delineated a technical trajectory for the integration of ML into their military frameworks. To bolster the structure and efficacy of their various military branches and units, there has been a concentrated deployment of numerous ML research endeavors. These initiatives encompass the study of sophisticated ML algorithms and the acceleration of artificial intelligence technology adaptation for intelligence processing, autonomous platforms, command and control infrastructures, and weapons systems. Forces across the globe are actively embedding ML technologies into a range of platforms-terrestrial, naval, aerial, space-faring, and cybernetic. This integration spans weaponry, networks, cognitive operations, and additional systems. Furthermore, this article reviews the incorporation within the sphere of military human–machine interaction in the Russia–Ukraine conflict. In this war, cyber human–machine interaction has become a pivotal arena of contention between Russia and Ukraine, with key levers that influence the conflict's course. In addition, the article examines the adoption of ML in prospective military functions such as, operations, intelligence gathering, networking, logistics, identification protocols, healthcare, data analysis trends, and other critical areas marked by current developments and trajectories. It also proffers a series of recommendations for the future integration of ML to inform strategic direction and research.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 6","pages":"753-761"},"PeriodicalIF":3.5,"publicationDate":"2024-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142691707","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-09DOI: 10.1109/THMS.2024.3462416
Hang Su;Francesco Jamal Sheiban;Wen Qi;Salih Ertug Ovur;Samer Alfayad
The increasingly pervasive usage of robotic surgery not only calls for advances in clinical application but also implies high availability for preliminary medical education using virtual reality. Virtual reality is currently upgrading medical education by presenting complicated medical information in an immersive and interactive way. A system that allows multiple users to observe and operate via simulated surgical platforms using wearable devices has become an efficient solution for teaching where a real surgical platform is not available. This article developed a bioinspired virtual reality toolkit (BioVRbot) for education and training in robot-assisted minimally invasive surgery. It allows multiple users to manipulate the robots working on cooperative virtual surgery using bioinspired control. The virtual reality scenario is implemented using unity and can be observed with independent virtual reality headsets. A MATLAB server is designed to manage robot motion planning of incremental teleoperation compliance with the remote center of motion constraints. Wearable sensorized gloves are adopted for continuous control of the tooltip and the gripper. Finally, the practical use of the developed surgical virtual system is demonstrated with cooperative operation tasks. It could be further spread into the classroom for preliminary education of robot-assisted surgery for early-stage medical students.
{"title":"A Bioinspired Virtual Reality Toolkit for Robot-Assisted Medical Application: BioVRbot","authors":"Hang Su;Francesco Jamal Sheiban;Wen Qi;Salih Ertug Ovur;Samer Alfayad","doi":"10.1109/THMS.2024.3462416","DOIUrl":"https://doi.org/10.1109/THMS.2024.3462416","url":null,"abstract":"The increasingly pervasive usage of robotic surgery not only calls for advances in clinical application but also implies high availability for preliminary medical education using virtual reality. Virtual reality is currently upgrading medical education by presenting complicated medical information in an immersive and interactive way. A system that allows multiple users to observe and operate via simulated surgical platforms using wearable devices has become an efficient solution for teaching where a real surgical platform is not available. This article developed a bioinspired virtual reality toolkit (BioVRbot) for education and training in robot-assisted minimally invasive surgery. It allows multiple users to manipulate the robots working on cooperative virtual surgery using bioinspired control. The virtual reality scenario is implemented using unity and can be observed with independent virtual reality headsets. A MATLAB server is designed to manage robot motion planning of incremental teleoperation compliance with the remote center of motion constraints. Wearable sensorized gloves are adopted for continuous control of the tooltip and the gripper. Finally, the practical use of the developed surgical virtual system is demonstrated with cooperative operation tasks. It could be further spread into the classroom for preliminary education of robot-assisted surgery for early-stage medical students.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 6","pages":"688-697"},"PeriodicalIF":3.5,"publicationDate":"2024-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142691708","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
While it is crucial for human-like avatars to perform co-speech gestures, existing approaches struggle to generate natural and realistic movements. In the present study, a novel transformer-based denoising diffusion model is proposed to generate co-speech gestures. Moreover, we introduce a practical sampling trick for diffusion models to maintain the continuity between the generated motion segments while improving the within-segment motion likelihood and naturalness. Our model can be used for online generation since it generates gestures for a short segment of speech, e.g., 2 s. We evaluate our model on two large-scale speech-gesture datasets with finger movements using objective measurements and a user study, showing that our model outperforms all other baselines. Our user study is based on the Metahuman platform in the Unreal Engine, a popular tool for creating human-like avatars and motions.
{"title":"Speech-Driven Gesture Generation Using Transformer-Based Denoising Diffusion Probabilistic Models","authors":"Bowen Wu;Chaoran Liu;Carlos Toshinori Ishi;Hiroshi Ishiguro","doi":"10.1109/THMS.2024.3456085","DOIUrl":"https://doi.org/10.1109/THMS.2024.3456085","url":null,"abstract":"While it is crucial for human-like avatars to perform co-speech gestures, existing approaches struggle to generate natural and realistic movements. In the present study, a novel transformer-based denoising diffusion model is proposed to generate co-speech gestures. Moreover, we introduce a practical sampling trick for diffusion models to maintain the continuity between the generated motion segments while improving the within-segment motion likelihood and naturalness. Our model can be used for online generation since it generates gestures for a short segment of speech, e.g., 2 s. We evaluate our model on two large-scale speech-gesture datasets with finger movements using objective measurements and a user study, showing that our model outperforms all other baselines. Our user study is based on the Metahuman platform in the Unreal Engine, a popular tool for creating human-like avatars and motions.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 6","pages":"733-742"},"PeriodicalIF":3.5,"publicationDate":"2024-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10712170","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142691706","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Robot-assisted catheterization offers a promising technique for cardiovascular interventions, addressing the limitations of manual interventional surgery, where precise tool manipulation is critical. In remote-control robotic systems, the lack of force feedback and imprecise navigation challenge cooperation between the surgeon and robot. This study proposes a manipulation-based evaluation framework to assess the cooperative performance between different operators and robot using kinesthetic, kinematic, and haptic data from multi-sensor technologies. The proposed evaluation framework achieves a recognition accuracy of 99.99% in assessing the cooperation between operator and robot. Additionally, the study investigates the impact of delay factors, considering no delay, constant delay, and variable delay, on cooperation characteristics. The findings suggest that variable delay contributes to improved cooperation performance between operator and robot in a primary-secondary isomorphic robotic system, compared to a constant delay factor. Furthermore, operators with experience in manual percutaneous coronary interventions exhibit significantly better cooperative manipulate on with the robot system than those without such experience, with respective synergy ratios of 89.66%, 90.28%, and 91.12% based on the three aspects of delay consideration. Moreover, the study explores interaction information, including distal force of tools-tissue and contact force of hand-control-ring, to understand how operators with different technical skills adjust their control strategy to prevent damage to the vascular vessel caused by excessive force while ensuring enough tension to navigate complex paths. The findings highlight the potential of variable delay to enhance cooperative control strategies in robotic catheterization systems, providing a basis for optimizing surgeon-robot collaboration in cardiovascular interventions.
{"title":"Analyzing Surgeon–Robot Cooperative Performance in Robot-Assisted Intravascular Catheterization","authors":"Wenjing Du;Guanlin Yi;Olatunji Mumini Omisore;Wenke Duan;Toluwanimi Oluwadra Akinyemi;Xingyu Chen;Jiang Liu;Boon-Giin Lee;Lei Wang","doi":"10.1109/THMS.2024.3452975","DOIUrl":"https://doi.org/10.1109/THMS.2024.3452975","url":null,"abstract":"Robot-assisted catheterization offers a promising technique for cardiovascular interventions, addressing the limitations of manual interventional surgery, where precise tool manipulation is critical. In remote-control robotic systems, the lack of force feedback and imprecise navigation challenge cooperation between the surgeon and robot. This study proposes a manipulation-based evaluation framework to assess the cooperative performance between different operators and robot using kinesthetic, kinematic, and haptic data from multi-sensor technologies. The proposed evaluation framework achieves a recognition accuracy of 99.99% in assessing the cooperation between operator and robot. Additionally, the study investigates the impact of delay factors, considering no delay, constant delay, and variable delay, on cooperation characteristics. The findings suggest that variable delay contributes to improved cooperation performance between operator and robot in a primary-secondary isomorphic robotic system, compared to a constant delay factor. Furthermore, operators with experience in manual percutaneous coronary interventions exhibit significantly better cooperative manipulate on with the robot system than those without such experience, with respective synergy ratios of 89.66%, 90.28%, and 91.12% based on the three aspects of delay consideration. Moreover, the study explores interaction information, including distal force of tools-tissue and contact force of hand-control-ring, to understand how operators with different technical skills adjust their control strategy to prevent damage to the vascular vessel caused by excessive force while ensuring enough tension to navigate complex paths. The findings highlight the potential of variable delay to enhance cooperative control strategies in robotic catheterization systems, providing a basis for optimizing surgeon-robot collaboration in cardiovascular interventions.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 6","pages":"698-710"},"PeriodicalIF":3.5,"publicationDate":"2024-10-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10705684","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142691687","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-27DOI: 10.1109/THMS.2024.3458905
Lingzhou Yu;Shaoping Bai
A major challenge in the lower limb exoskeleton for walking assistance is the adaptive gait control. In this article, a modified dynamic movement primitive (DMP) (MDMP) control is proposed to achieve gait adjustment with different assistance levels. This is achieved by inclusion of interaction forces in the formulation of DMP, which enables learning from physical human–robot interaction. A threshold force is introduced accounting for different levels of walking assistance from the exoskeleton. The MDMP is, thus, capable of generating adjustable gait and reshaping trajectories with data from the interaction force sensors. The experiments on five subjects show that the average differences between the human body and the exoskeleton are 4.13° and 1.92° on the hip and knee, respectively, with average interaction forces of 42.54 N and 26.36 N exerted on the subjects' thigh and shank. The results demonstrated that the MDMP method can effectively provide adjustable gait for walking assistance.
{"title":"A Modified Dynamic Movement Primitive Algorithm for Adaptive Gait Control of a Lower Limb Exoskeleton","authors":"Lingzhou Yu;Shaoping Bai","doi":"10.1109/THMS.2024.3458905","DOIUrl":"https://doi.org/10.1109/THMS.2024.3458905","url":null,"abstract":"A major challenge in the lower limb exoskeleton for walking assistance is the adaptive gait control. In this article, a modified dynamic movement primitive (DMP) (MDMP) control is proposed to achieve gait adjustment with different assistance levels. This is achieved by inclusion of interaction forces in the formulation of DMP, which enables learning from physical human–robot interaction. A threshold force is introduced accounting for different levels of walking assistance from the exoskeleton. The MDMP is, thus, capable of generating adjustable gait and reshaping trajectories with data from the interaction force sensors. The experiments on five subjects show that the average differences between the human body and the exoskeleton are 4.13° and 1.92° on the hip and knee, respectively, with average interaction forces of 42.54 N and 26.36 N exerted on the subjects' thigh and shank. The results demonstrated that the MDMP method can effectively provide adjustable gait for walking assistance.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 6","pages":"778-787"},"PeriodicalIF":3.5,"publicationDate":"2024-09-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142691689","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-19DOI: 10.1109/THMS.2024.3458771
{"title":"Present a World of Opportunity","authors":"","doi":"10.1109/THMS.2024.3458771","DOIUrl":"https://doi.org/10.1109/THMS.2024.3458771","url":null,"abstract":"","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":"54 5","pages":"631-631"},"PeriodicalIF":3.5,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10684407","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142246401","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：