Pub Date : 2022-05-23DOI: 10.1109/icra46639.2022.9811616
Dennis Mronga, Shivesh Kumar, F. Kirchner
Parallel mechanisms are becoming increasingly popular as subsystems in various robots due to their superior stiffness, payload-to-weight ratio, and dynamic properties. The serial connection of parallel subsystems leads to series-parallel hybrid robots, which are more difficult to model and control than serial or tree-type systems. At the same time, Whole-Body Control (WBC) has become the method of choice in the control of robots with redundant degrees of freedom, e.g., legged robots. However, most state-of-the-art WBC frameworks can only deal with serial or tree-type robot topologies. In this paper, we describe a computationally efficient framework for Whole-Body Control of series-parallel hybrid robots subjected to a large number of holonomic constraints. In contrast to existing WBC frameworks, our approach describes the optimization problem in the actuation space of a series-parallel robot, which provides better exploitation of the feasible workspace, higher accuracy, and more transparent behavior near singularities. We evaluate the proposed framework on two different humanoids with series-parallel architecture and compare its performance to a WBC approach for tree-type robots.
{"title":"Whole-Body Control of Series-Parallel Hybrid Robots","authors":"Dennis Mronga, Shivesh Kumar, F. Kirchner","doi":"10.1109/icra46639.2022.9811616","DOIUrl":"https://doi.org/10.1109/icra46639.2022.9811616","url":null,"abstract":"Parallel mechanisms are becoming increasingly popular as subsystems in various robots due to their superior stiffness, payload-to-weight ratio, and dynamic properties. The serial connection of parallel subsystems leads to series-parallel hybrid robots, which are more difficult to model and control than serial or tree-type systems. At the same time, Whole-Body Control (WBC) has become the method of choice in the control of robots with redundant degrees of freedom, e.g., legged robots. However, most state-of-the-art WBC frameworks can only deal with serial or tree-type robot topologies. In this paper, we describe a computationally efficient framework for Whole-Body Control of series-parallel hybrid robots subjected to a large number of holonomic constraints. In contrast to existing WBC frameworks, our approach describes the optimization problem in the actuation space of a series-parallel robot, which provides better exploitation of the feasible workspace, higher accuracy, and more transparent behavior near singularities. We evaluate the proposed framework on two different humanoids with series-parallel architecture and compare its performance to a WBC approach for tree-type robots.","PeriodicalId":341244,"journal":{"name":"2022 International Conference on Robotics and Automation (ICRA)","volume":"8 6","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120998000","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-05-23DOI: 10.1109/icra46639.2022.9811795
Eunhye Kim, Masaru Takeuchi, Takuto Nomura, Y. Hasegawa, Qiang Huang, Toshio Fukuda
In this paper, we fabricated a soft sensor based on PEDOT:PSS for thin film structure. The developed soft sensor can measure the contraction force at real time to be embedded in a modular bio-actuator [1]. The modular actuator generated contraction forces at 0.3 mN when applying electric pulse stimulation. To measure millinewton contraction forces and make a built in sensor, we fabricated a soft sensor using PEDOT:PSS-PDMS film. To verify that the sensor can measure the force of the actuator and can be integrated to the actuator, we analyzed characteristic of the sensor. First, we measure Young's modulus of the sensor and compare them with the bio-actuator. From the previous research [2], the Young's modulus of the bio-actuator and sensor were 45.8 kPa and 165 kPa, respectively. In addition, we simulated the sensors to estimate the change of the displacement according to the applied force. Next, we have experiments by stretching sensors using stepping motor to measure the resistance change of the sensor. From the simulation data, the displacement change is 23 µm when applying 0.3 mN of forces and then we detect the displacement change smaller than is 20 µm from the experiments. Finally, we analyzed the movement of the bio-actuator when applying stimulation using high speed camera and time response of the developed sensor. The actuator was contracted to the maximum after 150 ms from the electrical stimulation and the sensor detected the repeated motion at 10 Hz without time delay. As a result, the proposed sensor can measure the force of bioactuator at real time.
{"title":"Fabrication of PEDOT:PSS based Soft Sensor for Feedback Control of Modular Bio-actuator","authors":"Eunhye Kim, Masaru Takeuchi, Takuto Nomura, Y. Hasegawa, Qiang Huang, Toshio Fukuda","doi":"10.1109/icra46639.2022.9811795","DOIUrl":"https://doi.org/10.1109/icra46639.2022.9811795","url":null,"abstract":"In this paper, we fabricated a soft sensor based on PEDOT:PSS for thin film structure. The developed soft sensor can measure the contraction force at real time to be embedded in a modular bio-actuator [1]. The modular actuator generated contraction forces at 0.3 mN when applying electric pulse stimulation. To measure millinewton contraction forces and make a built in sensor, we fabricated a soft sensor using PEDOT:PSS-PDMS film. To verify that the sensor can measure the force of the actuator and can be integrated to the actuator, we analyzed characteristic of the sensor. First, we measure Young's modulus of the sensor and compare them with the bio-actuator. From the previous research [2], the Young's modulus of the bio-actuator and sensor were 45.8 kPa and 165 kPa, respectively. In addition, we simulated the sensors to estimate the change of the displacement according to the applied force. Next, we have experiments by stretching sensors using stepping motor to measure the resistance change of the sensor. From the simulation data, the displacement change is 23 µm when applying 0.3 mN of forces and then we detect the displacement change smaller than is 20 µm from the experiments. Finally, we analyzed the movement of the bio-actuator when applying stimulation using high speed camera and time response of the developed sensor. The actuator was contracted to the maximum after 150 ms from the electrical stimulation and the sensor detected the repeated motion at 10 Hz without time delay. As a result, the proposed sensor can measure the force of bioactuator at real time.","PeriodicalId":341244,"journal":{"name":"2022 International Conference on Robotics and Automation (ICRA)","volume":"397 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116654142","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-05-23DOI: 10.1109/icra46639.2022.9811829
Patrick Geneva, G. Huang
We revisit the problem of efficiently leveraging prior map information within a visual-inertial estimation framework. The use of traditional landmark-based maps with 2D-to-3D measurements along with the recently introduced keyframe-based maps with 2D-to-2D measurements are inves-tigated. The full joint estimation of the prior map is compared within a visual-inertial simulator to the Schmidt-Kalman filter (SKF) and measurement inflation methods in terms of their computational complexity, consistency, accuracy, and memory usage. This study shows that the SKF can enable efficient and consistent estimation for small workspace scenarios and the use of 2D-to-3D landmark maps have the highest levels of accuracy. Keyframe-based 2D-to-2D maps can reduce the required state size while still enabling accuracy gains. Finally, we show that measurement inflation methods, after tuning, can be accurate and efficient for large-scale environments if the guarantee of consistency is relaxed.
{"title":"Map-based Visual-Inertial Localization: A Numerical Study","authors":"Patrick Geneva, G. Huang","doi":"10.1109/icra46639.2022.9811829","DOIUrl":"https://doi.org/10.1109/icra46639.2022.9811829","url":null,"abstract":"We revisit the problem of efficiently leveraging prior map information within a visual-inertial estimation framework. The use of traditional landmark-based maps with 2D-to-3D measurements along with the recently introduced keyframe-based maps with 2D-to-2D measurements are inves-tigated. The full joint estimation of the prior map is compared within a visual-inertial simulator to the Schmidt-Kalman filter (SKF) and measurement inflation methods in terms of their computational complexity, consistency, accuracy, and memory usage. This study shows that the SKF can enable efficient and consistent estimation for small workspace scenarios and the use of 2D-to-3D landmark maps have the highest levels of accuracy. Keyframe-based 2D-to-2D maps can reduce the required state size while still enabling accuracy gains. Finally, we show that measurement inflation methods, after tuning, can be accurate and efficient for large-scale environments if the guarantee of consistency is relaxed.","PeriodicalId":341244,"journal":{"name":"2022 International Conference on Robotics and Automation (ICRA)","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124968082","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-05-23DOI: 10.48550/arXiv.2207.01158
Danpeng Chen, Shuai Wang, Wei-Yu Xie, Shangjin Zhai, Nan Wang, H. Bao, Guofeng Zhang
In this paper, we propose a tightly-coupled SLAM system fused with RGB, Depth, IMU and structured plane information. Traditional sparse points based SLAM systems always maintain a mass of map points to model the environment. Huge number of map points bring us a high computational complexity, making it difficult to be deployed on mobile devices. On the other hand, planes are common structures in man-made environment especially in indoor environments. We usually can use a small number of planes to represent a large scene. So the main purpose of this article is to decrease the high complexity of sparse points based SLAM. We build a lightweight back-end map which consists of a few planes and map points to achieve efficient bundle adjustment (BA) with an equal or better accuracy. We use homography constraints to eliminate the parameters of numerous plane points in the optimization and reduce the complexity of BA. We separate the parameters and measurements in homography and point-to-plane constraints and compress the measurements part to further effectively im-prove the speed of BA. We also integrate the plane information into the whole system to realize robust planar feature extraction, data association, and global consistent planar reconstruction. Finally, we perform an ablation study and compare our method with similar methods in simulation and real environment data. Our system achieves obvious advantages in accuracy and efficiency. Even if the plane parameters are involved in the optimization, we effectively simplify the back-end map by using planar structures. The global bundle adjustment is nearly 2 times faster than the sparse points based SLAM algorithm.
{"title":"VIP-SLAM: An Efficient Tightly-Coupled RGB-D Visual Inertial Planar SLAM","authors":"Danpeng Chen, Shuai Wang, Wei-Yu Xie, Shangjin Zhai, Nan Wang, H. Bao, Guofeng Zhang","doi":"10.48550/arXiv.2207.01158","DOIUrl":"https://doi.org/10.48550/arXiv.2207.01158","url":null,"abstract":"In this paper, we propose a tightly-coupled SLAM system fused with RGB, Depth, IMU and structured plane information. Traditional sparse points based SLAM systems always maintain a mass of map points to model the environment. Huge number of map points bring us a high computational complexity, making it difficult to be deployed on mobile devices. On the other hand, planes are common structures in man-made environment especially in indoor environments. We usually can use a small number of planes to represent a large scene. So the main purpose of this article is to decrease the high complexity of sparse points based SLAM. We build a lightweight back-end map which consists of a few planes and map points to achieve efficient bundle adjustment (BA) with an equal or better accuracy. We use homography constraints to eliminate the parameters of numerous plane points in the optimization and reduce the complexity of BA. We separate the parameters and measurements in homography and point-to-plane constraints and compress the measurements part to further effectively im-prove the speed of BA. We also integrate the plane information into the whole system to realize robust planar feature extraction, data association, and global consistent planar reconstruction. Finally, we perform an ablation study and compare our method with similar methods in simulation and real environment data. Our system achieves obvious advantages in accuracy and efficiency. Even if the plane parameters are involved in the optimization, we effectively simplify the back-end map by using planar structures. The global bundle adjustment is nearly 2 times faster than the sparse points based SLAM algorithm.","PeriodicalId":341244,"journal":{"name":"2022 International Conference on Robotics and Automation (ICRA)","volume":"63 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116719033","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-05-23DOI: 10.1109/icra46639.2022.9811769
Yiheng Han, I. Zhan, Wang Zhao, Yong-Jin Liu
Next best view (NBV) is a technology that finds the best view sequence for sensor to perform scanning based on partial information, which is the core part for robot active reconstruction. Traditional works are mostly based on the evaluation of candidate views through time-consuming volu-metric transformation and ray casting, which heavily limits the applications of NBV. Recent deep learning based NBV methods aim to approximately learn the evaluation function by large-scale training, and improve both the effectiveness and efficiency of NBV. However, these methods force the network to regress the exact groundtruth value of each candidate view, which is much harder than simply ranking all the candidate views. Besides, most previous NBV works assume perfect sensing and perform in simulation environments, lacking real application abilities. In this paper, we propose a novel double branch NBV network, DB-NBV, to utilize the ranking process together with the evaluation process. We further design a real NBV robot and a pipeline to conduct real active reconstruction. Experiments on both simulation and real robot show that our method achieves the best performance and can be applied to real application with high accuracy and speed.
{"title":"A Double Branch Next-Best-View Network and Novel Robot System for Active Object Reconstruction","authors":"Yiheng Han, I. Zhan, Wang Zhao, Yong-Jin Liu","doi":"10.1109/icra46639.2022.9811769","DOIUrl":"https://doi.org/10.1109/icra46639.2022.9811769","url":null,"abstract":"Next best view (NBV) is a technology that finds the best view sequence for sensor to perform scanning based on partial information, which is the core part for robot active reconstruction. Traditional works are mostly based on the evaluation of candidate views through time-consuming volu-metric transformation and ray casting, which heavily limits the applications of NBV. Recent deep learning based NBV methods aim to approximately learn the evaluation function by large-scale training, and improve both the effectiveness and efficiency of NBV. However, these methods force the network to regress the exact groundtruth value of each candidate view, which is much harder than simply ranking all the candidate views. Besides, most previous NBV works assume perfect sensing and perform in simulation environments, lacking real application abilities. In this paper, we propose a novel double branch NBV network, DB-NBV, to utilize the ranking process together with the evaluation process. We further design a real NBV robot and a pipeline to conduct real active reconstruction. Experiments on both simulation and real robot show that our method achieves the best performance and can be applied to real application with high accuracy and speed.","PeriodicalId":341244,"journal":{"name":"2022 International Conference on Robotics and Automation (ICRA)","volume":"55 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125117352","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-05-23DOI: 10.1109/icra46639.2022.9811369
Chenxin Lyu, Zhihao Fan, Yuanbo Bi, Zheng Zeng, L. Lian
Underwater thermocline, common in the lakes and ocean, plays a vital role in meteorological forecasting in the ocean and lakes dynamics research. This letter proposes a method for rapid and multipoint observation of thermocline variations with time and space using an airdropped micro-profiler array, named the DRAGONFLY system. It comprises specially designed disposable low-cost micro-profilers, a general unmanned aerial carrier platform, and a ground control system. This system can conduct periodic profile observations at a single point or quickly survey a large area. A series of experiments to characterize the micro-profiler and the DRAGONFLY system were conducted in Qiandao Lake, China. We demonstrate the developed system with data from field experiments, which show very high flexibility, and feasibility to observe the lake thermocline, implying potential applications in ocean transient phenomena observation.
{"title":"DRAGONFLY: a UAV Rapidly Deployed Micro-Profiler Array for Underwater Thermocline Observation","authors":"Chenxin Lyu, Zhihao Fan, Yuanbo Bi, Zheng Zeng, L. Lian","doi":"10.1109/icra46639.2022.9811369","DOIUrl":"https://doi.org/10.1109/icra46639.2022.9811369","url":null,"abstract":"Underwater thermocline, common in the lakes and ocean, plays a vital role in meteorological forecasting in the ocean and lakes dynamics research. This letter proposes a method for rapid and multipoint observation of thermocline variations with time and space using an airdropped micro-profiler array, named the DRAGONFLY system. It comprises specially designed disposable low-cost micro-profilers, a general unmanned aerial carrier platform, and a ground control system. This system can conduct periodic profile observations at a single point or quickly survey a large area. A series of experiments to characterize the micro-profiler and the DRAGONFLY system were conducted in Qiandao Lake, China. We demonstrate the developed system with data from field experiments, which show very high flexibility, and feasibility to observe the lake thermocline, implying potential applications in ocean transient phenomena observation.","PeriodicalId":341244,"journal":{"name":"2022 International Conference on Robotics and Automation (ICRA)","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114265945","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-05-23DOI: 10.1109/icra46639.2022.9811727
S. Monteleone, F. Negrello, G. Grioli, M. Catalano
Compliant actuation bestows robots with the ability to cope with unstructured environments, move with agility, and interact safely with humans at the expense of reduced tracking accuracy. The inclusion of dampening components aims to reduce oscillatory dynamics and partially restore precision without sacrificing the previously obtained characteristics. This paper introduces the concept and design of a novel damped compliant actuator suitable for building multi-degree of freedom systems. The proposed unit has a unique actuator topology that has never been seen before in the literature. The gearbox is used as a differential component, allowing the design of compact units without giving up safety and accuracy enhancements. We present and analyze the actuator's model and experimentally characterize the actuator prototype and the elastic and damping component.
{"title":"dSEDA: a Differential Series Elastic Damped Actuator","authors":"S. Monteleone, F. Negrello, G. Grioli, M. Catalano","doi":"10.1109/icra46639.2022.9811727","DOIUrl":"https://doi.org/10.1109/icra46639.2022.9811727","url":null,"abstract":"Compliant actuation bestows robots with the ability to cope with unstructured environments, move with agility, and interact safely with humans at the expense of reduced tracking accuracy. The inclusion of dampening components aims to reduce oscillatory dynamics and partially restore precision without sacrificing the previously obtained characteristics. This paper introduces the concept and design of a novel damped compliant actuator suitable for building multi-degree of freedom systems. The proposed unit has a unique actuator topology that has never been seen before in the literature. The gearbox is used as a differential component, allowing the design of compact units without giving up safety and accuracy enhancements. We present and analyze the actuator's model and experimentally characterize the actuator prototype and the elastic and damping component.","PeriodicalId":341244,"journal":{"name":"2022 International Conference on Robotics and Automation (ICRA)","volume":"31 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125081359","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-05-23DOI: 10.1109/icra46639.2022.9811923
Pablo López-García, Stein Crispel, A. Varadharajan, Elias Saerens, T. Verstraten, B. Vanderborght, D. Lefeber
Robotic engineers face major challenges to solve the complex actuation needs of Human-Robot Collaboration with existing act robotic gearboxes. Available technologies comprise high-ratio Planetary Gearheads, Cycloid Drives and Harmonic Drives, inherited from conventional industrial robotics. Alternative approaches include Direct-Drive and Quasi Direct-Drive actuation strategies, which propose to cancel or substantially reduce gear ratio, in order to minimize reflected inertia and attain enough backdrivability for collaborative tasks. This paper presents the proof-of-concept validation of a novel high-ratio, Wolfrom-based, gearbox technology that follows a different approach to attain the same objective. Testing five different gearbox prototypes, we confirm the ability of the R2poweR technology to improve efficiency and backdrivability while retaining the weight and control advantages derived from the use of high reduction ratios. The result is a highly efficient, backdrivable, high-ratio gearbox with exciting Huma-Robot Collaboration potential.
{"title":"R2poweR: The Proof-of-Concept of a Backdrivable, High-Ratio Gearbox for Human-Robot Collaboration*","authors":"Pablo López-García, Stein Crispel, A. Varadharajan, Elias Saerens, T. Verstraten, B. Vanderborght, D. Lefeber","doi":"10.1109/icra46639.2022.9811923","DOIUrl":"https://doi.org/10.1109/icra46639.2022.9811923","url":null,"abstract":"Robotic engineers face major challenges to solve the complex actuation needs of Human-Robot Collaboration with existing act robotic gearboxes. Available technologies comprise high-ratio Planetary Gearheads, Cycloid Drives and Harmonic Drives, inherited from conventional industrial robotics. Alternative approaches include Direct-Drive and Quasi Direct-Drive actuation strategies, which propose to cancel or substantially reduce gear ratio, in order to minimize reflected inertia and attain enough backdrivability for collaborative tasks. This paper presents the proof-of-concept validation of a novel high-ratio, Wolfrom-based, gearbox technology that follows a different approach to attain the same objective. Testing five different gearbox prototypes, we confirm the ability of the R2poweR technology to improve efficiency and backdrivability while retaining the weight and control advantages derived from the use of high reduction ratios. The result is a highly efficient, backdrivable, high-ratio gearbox with exciting Huma-Robot Collaboration potential.","PeriodicalId":341244,"journal":{"name":"2022 International Conference on Robotics and Automation (ICRA)","volume":"70 ","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114002775","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Due to overly relying on appearance information or adopting direct static feature fusion, most of the existing action recognition methods based on multi-modality have poor robustness and insufficient consideration of modality differences. To address these problems, we propose a two-stream adaptive weight integration network with a three-dimensional parallel attention module, PA-AWCNN. Firstly, a three-dimensional Parallel Attention (PA) module is proposed to effectively extract features of spatial, temporal and channel dimensions and reduce the cross-dimensional interference, to achieve better robustness. Secondly, a Common Feature-driven (CFD) feature integration module is proposed to dynamically integrate appearance and depth features with adaptive weights, utilizing modality differences to redeem the lack of each feature, thereby balance the influence of both. The proposed PA-AW CNN uses the representative integrated feature generated by attention enhancement and feature integration for action recognition; it can not only get higher recognition accuracy but also improve the performance of distinguishing similar actions. Experiments illustrate that the proposed method achieves com-parable performances to state-of-the-art methods and obtains the accuracy of 92.76% and 95.65% on NTU RGB+D Dataset and SBU Kinect Interaction Dataset, respectively. The code is publicly available at: https://github.com/Luu-Yao/PA-AWCNN.
{"title":"PA-AWCNN: Two-stream Parallel Attention Adaptive Weight Network for RGB-D Action Recognition","authors":"Lu Yao, Sheng Liu, Chaonan Li, Siyu Zou, Shengyong Chen, Diyi Guan","doi":"10.1109/icra46639.2022.9811995","DOIUrl":"https://doi.org/10.1109/icra46639.2022.9811995","url":null,"abstract":"Due to overly relying on appearance information or adopting direct static feature fusion, most of the existing action recognition methods based on multi-modality have poor robustness and insufficient consideration of modality differences. To address these problems, we propose a two-stream adaptive weight integration network with a three-dimensional parallel attention module, PA-AWCNN. Firstly, a three-dimensional Parallel Attention (PA) module is proposed to effectively extract features of spatial, temporal and channel dimensions and reduce the cross-dimensional interference, to achieve better robustness. Secondly, a Common Feature-driven (CFD) feature integration module is proposed to dynamically integrate appearance and depth features with adaptive weights, utilizing modality differences to redeem the lack of each feature, thereby balance the influence of both. The proposed PA-AW CNN uses the representative integrated feature generated by attention enhancement and feature integration for action recognition; it can not only get higher recognition accuracy but also improve the performance of distinguishing similar actions. Experiments illustrate that the proposed method achieves com-parable performances to state-of-the-art methods and obtains the accuracy of 92.76% and 95.65% on NTU RGB+D Dataset and SBU Kinect Interaction Dataset, respectively. The code is publicly available at: https://github.com/Luu-Yao/PA-AWCNN.","PeriodicalId":341244,"journal":{"name":"2022 International Conference on Robotics and Automation (ICRA)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122599700","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-05-23DOI: 10.1109/icra46639.2022.9812180
Kecheng Shi, Rui Huang, Fengjun Mu, Zhinan Peng, Ke Huang, Y. Qin, Xiao Yang, Hong Cheng
Despite the advances in the field of human-robot interface (HRI) based on biological neural signal, the use of the sole electroencephalography (EEG) signal to help robotic exoskeleton predict the limb movement is currently no mature in rehabilitation training, due to its unreliability. Multimodal HRI represents a very recent solution to enhance the performance of single modal HRI. These HRI normally include the EEG signal with surface electromyography (sEMG) signal. However, their use for the lower limb movement prediction in hemiplegia is still limited, and the deep fusion feature of sEMG and EEG signal is ignored. This paper proposes a Dense co-attention mechanism-based Multimodal Enhance fusion Network (DMEFNet) for the lower limb movement prediction in hemiplegia. The DMEFNet can realize the mapping and deep fusion between the sEMG and EEG signal features and get a high accuracy movement prediction of the lower limbs. A sEMG and EEG data acquisition experiment and an incomplete asynchronous data collection paradigm are designed to verify the effectiveness of DMEFNet. The experimental results show that DMEFNet has a good movement prediction performance in both within-subject and cross-subject situations, reaching an accuracy of 82.96% and 88.44% respectively.
{"title":"A Novel Multimodal Human-Exoskeleton Interface Based on EEG and sEMG Activity for Rehabilitation Training","authors":"Kecheng Shi, Rui Huang, Fengjun Mu, Zhinan Peng, Ke Huang, Y. Qin, Xiao Yang, Hong Cheng","doi":"10.1109/icra46639.2022.9812180","DOIUrl":"https://doi.org/10.1109/icra46639.2022.9812180","url":null,"abstract":"Despite the advances in the field of human-robot interface (HRI) based on biological neural signal, the use of the sole electroencephalography (EEG) signal to help robotic exoskeleton predict the limb movement is currently no mature in rehabilitation training, due to its unreliability. Multimodal HRI represents a very recent solution to enhance the performance of single modal HRI. These HRI normally include the EEG signal with surface electromyography (sEMG) signal. However, their use for the lower limb movement prediction in hemiplegia is still limited, and the deep fusion feature of sEMG and EEG signal is ignored. This paper proposes a Dense co-attention mechanism-based Multimodal Enhance fusion Network (DMEFNet) for the lower limb movement prediction in hemiplegia. The DMEFNet can realize the mapping and deep fusion between the sEMG and EEG signal features and get a high accuracy movement prediction of the lower limbs. A sEMG and EEG data acquisition experiment and an incomplete asynchronous data collection paradigm are designed to verify the effectiveness of DMEFNet. The experimental results show that DMEFNet has a good movement prediction performance in both within-subject and cross-subject situations, reaching an accuracy of 82.96% and 88.44% respectively.","PeriodicalId":341244,"journal":{"name":"2022 International Conference on Robotics and Automation (ICRA)","volume":"40 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131584176","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：