Pub Date : 2024-03-18DOI: 10.1109/THMS.2024.3370582
Fengwei Gu;Jun Lu;Chengtao Cai;Qidan Zhu;Zhaojie Ju
In complex environments, trackers are extremely susceptible to some interference factors, such as fast motions, occlusion, and scale changes, which result in poor tracking performance. The reason is that trackers cannot sufficiently utilize the target feature information in these cases. Therefore, it has become a particularly critical issue in the field of visual tracking to utilize the target feature information efficiently. In this article, a composite transformer involving spatiotemporal features is proposed to achieve robust visual tracking. Our method develops a novel toroidal transformer to fully integrate features while designing a template refresh mechanism to provide temporal features efficiently. Combined with the hybrid attention mechanism, the composite of temporal and spatial feature information is more conducive to mining feature associations between the template and search region than a single feature. To further correlate the global information, the proposed method adopts a closed-loop structure of the toroidal transformer formed by the cross-feature fusion head to integrate features. Moreover, the designed score head is used as a basis for judging whether the template is refreshed. Ultimately, the proposed tracker can achieve the tracking task only through a simple network framework, which especially simplifies the existing tracking architectures. Experiments show that the proposed tracker outperforms extensive state-of-the-art methods on seven benchmarks at a real-time speed of 56.5 fps.
{"title":"RTSformer: A Robust Toroidal Transformer With Spatiotemporal Features for Visual Tracking","authors":"Fengwei Gu;Jun Lu;Chengtao Cai;Qidan Zhu;Zhaojie Ju","doi":"10.1109/THMS.2024.3370582","DOIUrl":"10.1109/THMS.2024.3370582","url":null,"abstract":"In complex environments, trackers are extremely susceptible to some interference factors, such as fast motions, occlusion, and scale changes, which result in poor tracking performance. The reason is that trackers cannot sufficiently utilize the target feature information in these cases. Therefore, it has become a particularly critical issue in the field of visual tracking to utilize the target feature information efficiently. In this article, a composite transformer involving spatiotemporal features is proposed to achieve robust visual tracking. Our method develops a novel toroidal transformer to fully integrate features while designing a template refresh mechanism to provide temporal features efficiently. Combined with the hybrid attention mechanism, the composite of temporal and spatial feature information is more conducive to mining feature associations between the template and search region than a single feature. To further correlate the global information, the proposed method adopts a closed-loop structure of the toroidal transformer formed by the cross-feature fusion head to integrate features. Moreover, the designed score head is used as a basis for judging whether the template is refreshed. Ultimately, the proposed tracker can achieve the tracking task only through a simple network framework, which especially simplifies the existing tracking architectures. Experiments show that the proposed tracker outperforms extensive state-of-the-art methods on seven benchmarks at a real-time speed of 56.5 fps.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-03-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140166154","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-18DOI: 10.1109/THMS.2024.3371099
Balint K. Hodossy;Annika S. Guez;Shibo Jing;Weiguang Huo;Ravi Vaidyanathan;Dario Farina
To control wearable robotic systems, it is critical to obtain a prediction of the user's motion intent with high accuracy. Surface electromyography (sEMG) recordings have often been used as inputs for these devices, however bipolar sEMG electrodes are highly sensitive to their location. Positional shifts of electrodes after training gait prediction models can therefore result in severe performance degradation. This study uses high-density sEMG (HD-sEMG) electrodes to simulate various bipolar electrode signals from four leg muscles during steady-state walking. The bipolar signals were ranked based on the consistency of the corresponding sEMG envelope's activity and timing across gait cycles. The locations were then compared by evaluating the performance of an offline temporal convolutional network (TCN) that mapped sEMG signals to knee angles. The results showed that electrode locations with consistent sEMG envelopes resulted in greater prediction accuracy compared to hand-aligned placements ( p < 0.01). However, performance gains through this process were limited, and did not resolve the position shift issue. Instead of training a model for a single location, we showed that randomly sampling bipolar combinations across the HD-sEMG grid during training mitigated this effect. Models trained with this method generalized over all positions, and achieved 70% less prediction error than location specific models over the entire area of the grid. Therefore, the use of HD-sEMG grids to build training datasets could enable the development of models robust to spatial variations, and reduce the impact of muscle-specific electrode placement on accuracy.
{"title":"Leveraging High-Density EMG to Investigate Bipolar Electrode Placement for Gait Prediction Models","authors":"Balint K. Hodossy;Annika S. Guez;Shibo Jing;Weiguang Huo;Ravi Vaidyanathan;Dario Farina","doi":"10.1109/THMS.2024.3371099","DOIUrl":"10.1109/THMS.2024.3371099","url":null,"abstract":"To control wearable robotic systems, it is critical to obtain a prediction of the user's motion intent with high accuracy. Surface electromyography (sEMG) recordings have often been used as inputs for these devices, however bipolar sEMG electrodes are highly sensitive to their location. Positional shifts of electrodes after training gait prediction models can therefore result in severe performance degradation. This study uses high-density sEMG (HD-sEMG) electrodes to simulate various bipolar electrode signals from four leg muscles during steady-state walking. The bipolar signals were ranked based on the consistency of the corresponding sEMG envelope's activity and timing across gait cycles. The locations were then compared by evaluating the performance of an offline temporal convolutional network (TCN) that mapped sEMG signals to knee angles. The results showed that electrode locations with consistent sEMG envelopes resulted in greater prediction accuracy compared to hand-aligned placements (\u0000<italic>p</i>\u0000 < 0.01). However, performance gains through this process were limited, and did not resolve the position shift issue. Instead of training a model for a single location, we showed that randomly sampling bipolar combinations across the HD-sEMG grid during training mitigated this effect. Models trained with this method generalized over all positions, and achieved 70% less prediction error than location specific models over the entire area of the grid. Therefore, the use of HD-sEMG grids to build training datasets could enable the development of models robust to spatial variations, and reduce the impact of muscle-specific electrode placement on accuracy.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-03-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10474208","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140166273","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-17DOI: 10.1109/THMS.2024.3395153
Shailesh Vitthalrao Bhalerao;Ram Bilas Pachori
In visual object decoding, magnetoencephalogram (MEG) and electroencephalogram (EEG) activation patterns demonstrate the utmost discriminative cognitive analysis due to their multivariate oscillatory nature. However, high noise in the recorded EEG-MEG signals and subject-specific variability make it extremely difficult to classify subject's cognitive responses to different visual stimuli. The proposed method is a multivariate extension of the swarm sparse decomposition method (MSSDM) for multivariate pattern analysis of EEG-MEG-based visual activation signals. In comparison, it is an advanced technique for decomposing nonstationary multicomponent signals into a finite number of channel-aligned oscillatory components that significantly enhance visual activation-related sub-bands. The MSSDM method adopts multivariate swarm filtering and sparse spectrum to automatically deliver optimal frequency bands in channel-specific sparse spectrums, resulting in improved filter banks. By combining the advantages of the multivariate SSDM and Riemann's correlation-assisted fusion feature (RCFF), the MSSDM-RCFF algorithm is investigated to improve the visual object recognition ability of EEG-MEG signals. We have also proposed time–frequency representation based on MSSDM to analyze discriminative cognitive patterns of different visual object classes from multichannel EEG-MEG signals. A proposed MSSDM is evaluated on multivariate synthetic signals and multivariate EEG-MEG signals using five classifiers. The proposed fusion feature and linear discriminant analysis classifier-based framework outperformed all existing state-of-the-art methods used for visual object detection and achieved the highest accuracy of 86.42% using tenfold cross-validation on EEG-MEG multichannel signals.
{"title":"Automated Classification of Cognitive Visual Objects Using Multivariate Swarm Sparse Decomposition From Multichannel EEG-MEG Signals","authors":"Shailesh Vitthalrao Bhalerao;Ram Bilas Pachori","doi":"10.1109/THMS.2024.3395153","DOIUrl":"10.1109/THMS.2024.3395153","url":null,"abstract":"In visual object decoding, magnetoencephalogram (MEG) and electroencephalogram (EEG) activation patterns demonstrate the utmost discriminative cognitive analysis due to their multivariate oscillatory nature. However, high noise in the recorded EEG-MEG signals and subject-specific variability make it extremely difficult to classify subject's cognitive responses to different visual stimuli. The proposed method is a multivariate extension of the swarm sparse decomposition method (MSSDM) for multivariate pattern analysis of EEG-MEG-based visual activation signals. In comparison, it is an advanced technique for decomposing nonstationary multicomponent signals into a finite number of channel-aligned oscillatory components that significantly enhance visual activation-related sub-bands. The MSSDM method adopts multivariate swarm filtering and sparse spectrum to automatically deliver optimal frequency bands in channel-specific sparse spectrums, resulting in improved filter banks. By combining the advantages of the multivariate SSDM and Riemann's correlation-assisted fusion feature (RCFF), the MSSDM-RCFF algorithm is investigated to improve the visual object recognition ability of EEG-MEG signals. We have also proposed time–frequency representation based on MSSDM to analyze discriminative cognitive patterns of different visual object classes from multichannel EEG-MEG signals. A proposed MSSDM is evaluated on multivariate synthetic signals and multivariate EEG-MEG signals using five classifiers. The proposed fusion feature and linear discriminant analysis classifier-based framework outperformed all existing state-of-the-art methods used for visual object detection and achieved the highest accuracy of 86.42% using tenfold cross-validation on EEG-MEG multichannel signals.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":null,"pages":null},"PeriodicalIF":3.5,"publicationDate":"2024-03-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141063423","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-13DOI: 10.1109/THMS.2024.3389690
Anne Tryphosa Kamatham;Meena Alzamani;Allison Dockum;Siddhartha Sikdar;Biswarup Mukherjee
Ultrasound imaging or sonomyography has been found to be a robust modality for measuring muscle activity due to its ability to image deep-seated muscles directly while providing superior spatiotemporal specificity compared with surface electromyography-based techniques. Quantifying the morphological changes during muscle activity involves computationally expensive approaches for tracking muscle anatomical structures or extracting features from brightness-mode (B-mode) images and amplitude-mode signals. This article uses an offline regression convolutional neural network called SonoMyoNet to estimate continuous isometric force from sparse ultrasound scanlines. SonoMyoNet learns features from a few equispaced scanlines selected from B-mode images and utilizes the learned features to estimate continuous isometric force accurately. The performance of SonoMyoNet was evaluated by varying the number of scanlines to simulate the placement of multiple single-element ultrasound transducers in a wearable system. Results showed that SonoMyoNet could accurately predict isometric force with just four scanlines and is immune to speckle noise and shifts in the scanline location. Thus, the proposed network reduces the computational load involved in feature tracking algorithms and estimates muscle force from the global features of sparse ultrasound images.
与基于表面肌电图的技术相比，超声成像或超声肌电图能够直接对深层肌肉进行成像，同时具有更高的时空特异性，因此被认为是测量肌肉活动的可靠模式。要量化肌肉活动过程中的形态变化，需要采用计算昂贵的方法来跟踪肌肉解剖结构，或从亮度模式（B-mode）图像和振幅模式信号中提取特征。本文使用一种名为 SonoMyoNet 的离线回归卷积神经网络来估计稀疏超声扫描线中的连续等长力。SonoMyoNet 从 B 型图像中选取的几条等间距扫描线中学习特征，并利用所学特征准确估计连续等长力。通过改变扫描线的数量来模拟在可穿戴系统中放置多个单元素超声传感器的情况，对 SonoMyoNet 的性能进行了评估。结果表明，SonoMyoNet 只需四条扫描线就能准确预测等长力，并且不受斑点噪声和扫描线位置偏移的影响。因此，所提出的网络可减少特征跟踪算法的计算负荷，并根据稀疏超声图像的全局特征估算肌力。
{"title":"SonoMyoNet: A Convolutional Neural Network for Predicting Isometric Force From Highly Sparse Ultrasound Images","authors":"Anne Tryphosa Kamatham;Meena Alzamani;Allison Dockum;Siddhartha Sikdar;Biswarup Mukherjee","doi":"10.1109/THMS.2024.3389690","DOIUrl":"10.1109/THMS.2024.3389690","url":null,"abstract":"Ultrasound imaging or sonomyography has been found to be a robust modality for measuring muscle activity due to its ability to image deep-seated muscles directly while providing superior spatiotemporal specificity compared with surface electromyography-based techniques. Quantifying the morphological changes during muscle activity involves computationally expensive approaches for tracking muscle anatomical structures or extracting features from brightness-mode (B-mode) images and amplitude-mode signals. This article uses an offline regression convolutional neural network called SonoMyoNet to estimate continuous isometric force from sparse ultrasound scanlines. SonoMyoNet learns features from a few equispaced scanlines selected from B-mode images and utilizes the learned features to estimate continuous isometric force accurately. The performance of SonoMyoNet was evaluated by varying the number of scanlines to simulate the placement of multiple single-element ultrasound transducers in a wearable system. Results showed that SonoMyoNet could accurately predict isometric force with just four scanlines and is immune to speckle noise and shifts in the scanline location. Thus, the proposed network reduces the computational load involved in feature tracking algorithms and estimates muscle force from the global features of sparse ultrasound images.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140939947","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-07DOI: 10.1109/THMS.2024.3390415
Kankana Roy;Rajiv R. Sahay
Robust segmentation of hands in a cluttered environment for hand gesture recognition has remained a challenge in computer vision. In this work, a two-stage gesture recognition framework is proposed. In the first stage, we segment hands using the proposed deep learning algorithm, and in the second stage, we use these segmented hands to classify gestures using a novel structurally incoherent nonnegative matrix factorization approach. We propose a new deep learning framework for hand segmentation called densely dilated U-Net. We exploit recently proposed dense blocks and dilated convolution layers in our work. To cope with the scarcity of labeled datasets we extend our densely dilated U-Net for semisupervised hand segmentation using hand bounding boxes as cues. We provide quantitative and qualitative evaluation of proposed hand segmentation model on several public hand segmentation datasets including EgoHands, GTEA, EYTH, EDSH, and HOF. Semisupervised segmentation results are also obtained on two hand detection datasets including VIVA and CVRR. As an extension of our work, we show semisupervised segmentation and gesture recognition results using segmented hands on NUS-II cluttered hand gesture dataset. To validate the efficiency of our semisupervised algorithm we evaluate it on OUHands dataset with real ground truth labels. For gesture classification, we propose a novel structurally incoherent nonnegative matrix factorization algorithm. We propose to use CNN features extracted from segmented images for nonnegative matrix factorization. Experimental results on NUS-II and OUHands datasets demonstrate that our two-stage approach for gesture recognition yields superior results.
{"title":"Hand Segmentation With Dense Dilated U-Net and Structurally Incoherent Nonnegative Matrix Factorization-Based Gesture Recognition","authors":"Kankana Roy;Rajiv R. Sahay","doi":"10.1109/THMS.2024.3390415","DOIUrl":"10.1109/THMS.2024.3390415","url":null,"abstract":"Robust segmentation of hands in a cluttered environment for hand gesture recognition has remained a challenge in computer vision. In this work, a two-stage gesture recognition framework is proposed. In the first stage, we segment hands using the proposed deep learning algorithm, and in the second stage, we use these segmented hands to classify gestures using a novel structurally incoherent nonnegative matrix factorization approach. We propose a new deep learning framework for hand segmentation called densely dilated U-Net. We exploit recently proposed dense blocks and dilated convolution layers in our work. To cope with the scarcity of labeled datasets we extend our densely dilated U-Net for semisupervised hand segmentation using hand bounding boxes as cues. We provide quantitative and qualitative evaluation of proposed hand segmentation model on several public hand segmentation datasets including EgoHands, GTEA, EYTH, EDSH, and HOF. Semisupervised segmentation results are also obtained on two hand detection datasets including VIVA and CVRR. As an extension of our work, we show semisupervised segmentation and gesture recognition results using segmented hands on NUS-II cluttered hand gesture dataset. To validate the efficiency of our semisupervised algorithm we evaluate it on OUHands dataset with real ground truth labels. For gesture classification, we propose a novel structurally incoherent nonnegative matrix factorization algorithm. We propose to use CNN features extracted from segmented images for nonnegative matrix factorization. Experimental results on NUS-II and OUHands datasets demonstrate that our two-stage approach for gesture recognition yields superior results.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-03-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140939745","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Flexible robotic systems change not only the production workflow as a whole but also the individual working conditions of their operators. The aim of this analysis of our study with more than 5900 participants was to get an overview of demographics, job characteristics, and health indicators of robot operators in comparison to nonrobotic machine operators and employees in Germany. We collected data by telephone interviews measuring technology use, stressors, and resources at work as well as health indicators. Results indicate systematic differences in working stressors and resources for robot users compared to other machine users as well as employees in general. In particular, the scope for decision-making at work was smaller for robot users, especially regarding the amount of work or the speed of work. Only isolated links could be found regarding the health indicators. The results therefore imply constant consideration of human factors to ensure productive as well as healthy working conditions with robots in modern industry.
{"title":"Working Conditions of Industrial Robot Operators–An Overview of Technology Dissemination, Job Characteristics, and Health Indicators in Modern Production Workplaces","authors":"Matthias Hartwig;Patricia Rosen;Sascha Wischniewski","doi":"10.1109/THMS.2024.3368525","DOIUrl":"10.1109/THMS.2024.3368525","url":null,"abstract":"Flexible robotic systems change not only the production workflow as a whole but also the individual working conditions of their operators. The aim of this analysis of our study with more than 5900 participants was to get an overview of demographics, job characteristics, and health indicators of robot operators in comparison to nonrobotic machine operators and employees in Germany. We collected data by telephone interviews measuring technology use, stressors, and resources at work as well as health indicators. Results indicate systematic differences in working stressors and resources for robot users compared to other machine users as well as employees in general. In particular, the scope for decision-making at work was smaller for robot users, especially regarding the amount of work or the speed of work. Only isolated links could be found regarding the health indicators. The results therefore imply constant consideration of human factors to ensure productive as well as healthy working conditions with robots in modern industry.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-03-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140055231","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-21DOI: 10.1109/THMS.2024.3360111
Zemin Liu;Qingsong Ai;Haojie Liu;Wei Meng;Quan Liu
The motor synergy pattern is an intrinsic characteristic found in natural human movements, particularly in the upper limb. It is essential to improve the multijoint coordination ability for stroke patients by integrating the synergy pattern into rehabilitation tasks and trajectory design. However, current robot-assisted rehabilitation systems tend to overlook the incorporation of a multijoint synergy model. This article proposes postural synergistic kernelized movement primitives (PSKMP) method for the human-like trajectory planning of robot-assisted upper limb rehabilitation. First, the demonstrated trajectory obtained from the motion capture system is subject to principal component analysis to extract postural synergies. Then, the PSKMP is proposed by kernelizing the postural synergistic subspaces with the kernel treatment to preserve human natural movement characteristics. Finally, the rehabilitation trajectory accord with human motion habits can be generated based on generalized postural synergistic subspaces. This approach has undergone practical validation on an upper limb rehabilitation robot, and the experimental results show that the proposed method enables the generation of human-like trajectories adapted to new task points, in accordance with the natural movement style of human. This method holds great significance in promoting the recovery of coordination ability of stroke patients.
{"title":"Human-Like Trajectory Planning Based on Postural Synergistic Kernelized Movement Primitives for Robot-Assisted Rehabilitation","authors":"Zemin Liu;Qingsong Ai;Haojie Liu;Wei Meng;Quan Liu","doi":"10.1109/THMS.2024.3360111","DOIUrl":"10.1109/THMS.2024.3360111","url":null,"abstract":"The motor synergy pattern is an intrinsic characteristic found in natural human movements, particularly in the upper limb. It is essential to improve the multijoint coordination ability for stroke patients by integrating the synergy pattern into rehabilitation tasks and trajectory design. However, current robot-assisted rehabilitation systems tend to overlook the incorporation of a multijoint synergy model. This article proposes postural synergistic kernelized movement primitives (PSKMP) method for the human-like trajectory planning of robot-assisted upper limb rehabilitation. First, the demonstrated trajectory obtained from the motion capture system is subject to principal component analysis to extract postural synergies. Then, the PSKMP is proposed by kernelizing the postural synergistic subspaces with the kernel treatment to preserve human natural movement characteristics. Finally, the rehabilitation trajectory accord with human motion habits can be generated based on generalized postural synergistic subspaces. This approach has undergone practical validation on an upper limb rehabilitation robot, and the experimental results show that the proposed method enables the generation of human-like trajectories adapted to new task points, in accordance with the natural movement style of human. This method holds great significance in promoting the recovery of coordination ability of stroke patients.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-02-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139950400","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-07DOI: 10.1109/THMS.2024.3356421
Jesus A. Orozco;Panagiotis Artemiadis
Trust is an essential building block of human civilization. However, when it relates to artificial systems, it has been a barrier to intelligent technology adoption in general. This article addresses the gap in determining levels of trust in scenarios that include humans interacting with a swarm of robots. Electroencephalography (EEG) recordings of the human observers of the different swarms allow for extracting specific EEG features related to different trust levels. Feature selection and machine learning methods comprise a classification system that would allow recognition of different levels of human trust in those human–swarm interaction scenarios. The results of this study suggest that EEG correlates of swarm trust exist and are distinguishable in machine learning feature classification with very high accuracy. Moreover, comparing common EEG features across all human subjects used in this study allows for the generalization of the classification method, providing solid evidence of specific areas and features of the human brain where activations are related to levels of human–swarm trust. This work has direct implications for effective human–machine teaming with applications to many fields, such as exploration, search and rescue operations, surveillance, environmental monitoring, and defense. In these applications, quantifying levels of human trust in the deployed swarm is of utmost importance because it can lead to swarm controllers that adapt their output based on the human's perceived trust level.
{"title":"Extracting Human Levels of Trust in Human–Swarm Interaction Using EEG Signals","authors":"Jesus A. Orozco;Panagiotis Artemiadis","doi":"10.1109/THMS.2024.3356421","DOIUrl":"10.1109/THMS.2024.3356421","url":null,"abstract":"Trust is an essential building block of human civilization. However, when it relates to artificial systems, it has been a barrier to intelligent technology adoption in general. This article addresses the gap in determining levels of trust in scenarios that include humans interacting with a swarm of robots. Electroencephalography (EEG) recordings of the human observers of the different swarms allow for extracting specific EEG features related to different trust levels. Feature selection and machine learning methods comprise a classification system that would allow recognition of different levels of human trust in those human–swarm interaction scenarios. The results of this study suggest that EEG correlates of swarm trust exist and are distinguishable in machine learning feature classification with very high accuracy. Moreover, comparing common EEG features across all human subjects used in this study allows for the generalization of the classification method, providing solid evidence of specific areas and features of the human brain where activations are related to levels of human–swarm trust. This work has direct implications for effective human–machine teaming with applications to many fields, such as exploration, search and rescue operations, surveillance, environmental monitoring, and defense. In these applications, quantifying levels of human trust in the deployed swarm is of utmost importance because it can lead to swarm controllers that adapt their output based on the human's perceived trust level.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-02-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139950346","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-02DOI: 10.1109/THMS.2024.3355474
Xin Lin;Shucong Yin;Hao Du;Yuquan Leng;Chenglong Fu
Suspended backpacks have been acknowledged for their advantages in load carriage, leading to the development of various designs aimed at enhancing their performance. However, current suspended backpacks typically possess fixed stiffness or limited adjustability, thereby limiting their adaptability to different load carriage tasks, such as varying walking speeds and load masses. This article introduced a suspended backpack design capable of modulating its stiffness over a wide range while maintaining a lightweight profile. The variable stiffness suspension (VSS) was integrated into the load frame of the suspended backpack and utilized a motor to adjust the stiffness by generating spring-like force based on the relative displacement between the load and the body. Experimental validation was conducted to assess the stiffness modulation of the suspended backpack. The VSS enabled the stiffness modulation of the suspended backpack ranging from 424 to 2182 N/m, which corresponded to the desired stiffness range for a 10–25 kg load at walking speeds for 3.5–6 km/h. Moreover, the mechanics of the carriers were analyzed to evaluate the impact of the suspended backpack on the individuals. Results showed that the designed VSS suspended backpack could reduce peak push-off force by 20.71% under the high working condition and energetic cost by 30.39% under the midworking condition. However, a tradeoff exists between minimizing the peak accelerative load force and energetic cost. The proposed design holds the potential for enhancing performance across various load carriage tasks, including human-in-the-loop energetic optimization.
{"title":"Design and Investigation of a Suspended Backpack With Wide-Range Variable Stiffness Suspension for Reducing Energetic Cost","authors":"Xin Lin;Shucong Yin;Hao Du;Yuquan Leng;Chenglong Fu","doi":"10.1109/THMS.2024.3355474","DOIUrl":"10.1109/THMS.2024.3355474","url":null,"abstract":"Suspended backpacks have been acknowledged for their advantages in load carriage, leading to the development of various designs aimed at enhancing their performance. However, current suspended backpacks typically possess fixed stiffness or limited adjustability, thereby limiting their adaptability to different load carriage tasks, such as varying walking speeds and load masses. This article introduced a suspended backpack design capable of modulating its stiffness over a wide range while maintaining a lightweight profile. The variable stiffness suspension (VSS) was integrated into the load frame of the suspended backpack and utilized a motor to adjust the stiffness by generating spring-like force based on the relative displacement between the load and the body. Experimental validation was conducted to assess the stiffness modulation of the suspended backpack. The VSS enabled the stiffness modulation of the suspended backpack ranging from 424 to 2182 N/m, which corresponded to the desired stiffness range for a 10–25 kg load at walking speeds for 3.5–6 km/h. Moreover, the mechanics of the carriers were analyzed to evaluate the impact of the suspended backpack on the individuals. Results showed that the designed VSS suspended backpack could reduce peak push-off force by 20.71% under the high working condition and energetic cost by 30.39% under the midworking condition. However, a tradeoff exists between minimizing the peak accelerative load force and energetic cost. The proposed design holds the potential for enhancing performance across various load carriage tasks, including human-in-the-loop energetic optimization.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-02-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139950499","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-01DOI: 10.1109/THMS.2024.3354413
Alejandro L. Callara;Gianluca Rho;Sara Condino;Vincenzo Ferrari;Enzo Pasquale Scilingo;Alberto Greco
Objective: The use of commercially available optical-see-through (OST) head-mounted displays (HMDs) in their own peripersonal space leads the user to experience two perception conflicts that deteriorate their performance in precision manual tasks: the vergence-accommodation conflict (VAC) and the focus rivalry. In this work, we aim characterizing for the first time the psychophysiological response associated with user's incorrect focus cues during the execution of an augmented reality (AR)-guided manual task with the Microsoft HoloLens OST-HMD. Methods: 21 subjects underwent to a “connecting-the-dots” experiment with and without the use of AR, and in both binocular and monocular conditions. For each condition, we quantified the changes in autonomic nervous system (ANS) activity of subjects by analyzing the electrodermal activity (EDA) and heart rate variability. Moreover, we analyzed the neural central correlates by means of power measures of brain activity and multivariate autoregressive measures of brain connectivity extracted from the electroencephalogram (EEG). Results: No statistically significant differences of ANS correlates were observed among tasks, although all EDA-related features varied between rest and task conditions. Conversely, significant differences among conditions were present in terms of EEG-power variations in the $mu$ (8–13) Hz and $beta$ (13–30) Hz bands. In addition, significant changes in the causal interactions of a brain network involved in motor movement and eye-hand coordination comprising the precentral gyrus, the precuneus, and the fusiform gyrus were observed. Conclusion: The physiological plausibility of our results suggest promising future applicability to investigate more complex scenarios, such as AR-guided surgery.
目的：在个人周围空间使用市售的光学透视（OST）头戴式显示器（HMD）会导致用户经历两种感知冲突，从而降低他们在精确手动任务中的表现：会聚-适应冲突（VAC）和焦点竞争。在这项工作中，我们旨在首次描述用户在使用微软 HoloLens OST-HMD 执行增强现实（AR）引导的手动任务时，与不正确的焦点提示相关的心理生理反应。方法：21 名受试者在使用或不使用 AR 的情况下，在双目和单目条件下进行了 "连线 "实验。在每种条件下，我们通过分析皮电活动（EDA）和心率变异性来量化受试者自律神经系统（ANS）活动的变化。此外，我们还通过大脑活动的功率测量和从脑电图（EEG）中提取的大脑连接的多变量自回归测量来分析神经中枢相关性。结果显示尽管所有与 EDA 相关的特征在休息和任务条件下均有不同，但在不同任务之间未观察到 ANS 相关因素存在统计学意义上的显著差异。相反，在 $mu$ (8-13) Hz 和 $beta$ (13-30) Hz 波段的脑电图功率变化方面，不同条件下存在明显差异。此外，由中央前回、楔前回和纺锤形回组成的大脑网络参与运动和眼手协调的因果相互作用也发生了重大变化。结论我们研究结果的生理学合理性表明，未来有望应用于研究更复杂的场景，如 AR 引导的外科手术。
{"title":"Behavioral, Peripheral, and Central Neural Correlates of Augmented Reality Guidance of Manual Tasks","authors":"Alejandro L. Callara;Gianluca Rho;Sara Condino;Vincenzo Ferrari;Enzo Pasquale Scilingo;Alberto Greco","doi":"10.1109/THMS.2024.3354413","DOIUrl":"10.1109/THMS.2024.3354413","url":null,"abstract":"Objective: The use of commercially available optical-see-through (OST) head-mounted displays (HMDs) in their own peripersonal space leads the user to experience two perception conflicts that deteriorate their performance in precision manual tasks: the vergence-accommodation conflict (VAC) and the focus rivalry. In this work, we aim characterizing for the first time the psychophysiological response associated with user's incorrect focus cues during the execution of an augmented reality (AR)-guided manual task with the Microsoft HoloLens OST-HMD. Methods: 21 subjects underwent to a “connecting-the-dots” experiment with and without the use of AR, and in both binocular and monocular conditions. For each condition, we quantified the changes in autonomic nervous system (ANS) activity of subjects by analyzing the electrodermal activity (EDA) and heart rate variability. Moreover, we analyzed the neural central correlates by means of power measures of brain activity and multivariate autoregressive measures of brain connectivity extracted from the electroencephalogram (EEG). Results: No statistically significant differences of ANS correlates were observed among tasks, although all EDA-related features varied between rest and task conditions. Conversely, significant differences among conditions were present in terms of EEG-power variations in the \u0000<inline-formula><tex-math>$mu$</tex-math></inline-formula>\u0000 (8–13) Hz and \u0000<inline-formula><tex-math>$beta$</tex-math></inline-formula>\u0000 (13–30) Hz bands. In addition, significant changes in the causal interactions of a brain network involved in motor movement and eye-hand coordination comprising the precentral gyrus, the precuneus, and the fusiform gyrus were observed. Conclusion: The physiological plausibility of our results suggest promising future applicability to investigate more complex scenarios, such as AR-guided surgery.","PeriodicalId":48916,"journal":{"name":"IEEE Transactions on Human-Machine Systems","volume":null,"pages":null},"PeriodicalIF":3.6,"publicationDate":"2024-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10418203","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139950356","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：