Pub Date : 2023-12-19DOI: 10.1007/s11370-023-00495-1
Abstract
Robot-assisted navigation is a perfect example of a class of applications requiring flexible control approaches. When the human is reliable, the robot should concede space to their initiative. When the human makes inappropriate choices the robot controller should kick-in guiding them towards safer paths. Shared authority control is a way to achieve this behaviour by deciding online how much of the authority should be given to the human and how much should be retained by the robot. An open problem is how to evaluate the appropriateness of the human’s choices. One possible way is to consider the deviation from an ideal path computed by the robot. This choice is certainly safe and efficient, but it emphasises the importance of the robot’s decision and relegates the human to a secondary role. In this paper, we propose a different paradigm: a human’s behaviour is correct if, at every time, it bears a close resemblance to what other humans do in similar situations. This idea is implemented through the combination of machine learning and adaptive control. The map of the environment is decomposed into a grid. In each cell, we classify the possible motions that the human executes. We use a neural network classifier to classify the current motion, and the probability score is used as a hyperparameter in the control to vary the amount of intervention. The experiments collected for the paper show the feasibility of the idea. A qualitative evaluation, done by surveying the users after they have tested the robot, shows that the participants preferred our control method over a state-of-the-art visco-elastic control.
{"title":"Humanising robot-assisted navigation","authors":"","doi":"10.1007/s11370-023-00495-1","DOIUrl":"https://doi.org/10.1007/s11370-023-00495-1","url":null,"abstract":"<h3>Abstract</h3> <p>Robot-assisted navigation is a perfect example of a class of applications requiring flexible control approaches. When the human is reliable, the robot should concede space to their initiative. When the human makes inappropriate choices the robot controller should kick-in guiding them towards safer paths. Shared authority control is a way to achieve this behaviour by deciding online how much of the authority should be given to the human and how much should be retained by the robot. An open problem is how to evaluate the appropriateness of the human’s choices. One possible way is to consider the deviation from an ideal path computed by the robot. This choice is certainly safe and efficient, but it emphasises the importance of the robot’s decision and relegates the human to a secondary role. In this paper, we propose a different paradigm: a human’s behaviour is correct if, at every time, it bears a close resemblance to what other humans do in similar situations. This idea is implemented through the combination of machine learning and adaptive control. The map of the environment is decomposed into a grid. In each cell, we classify the possible motions that the human executes. We use a neural network classifier to classify the current motion, and the probability score is used as a hyperparameter in the control to vary the amount of intervention. The experiments collected for the paper show the feasibility of the idea. A qualitative evaluation, done by surveying the users after they have tested the robot, shows that the participants preferred our control method over a state-of-the-art visco-elastic control.</p>","PeriodicalId":48813,"journal":{"name":"Intelligent Service Robotics","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2023-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138745010","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-19DOI: 10.1007/s11370-023-00496-0
Seongcheol Kim, Casey C. Bennett, Zachary Henkel, Jinjae Lee, Cedomir Stanojevic, Kenna Baugus, Cindy L. Bethel, Jennifer A. Piatt, Selma Šabanović
Deploying socially assistive robots (SARs) at home, such as robotic companion pets, can be useful for tracking behavioral and health-related changes in humans during lifestyle fluctuations over time, like those experienced during CoVID-19. However, a fundamental problem required when deploying autonomous agents such as SARs in people’s everyday living spaces is understanding how users interact with those robots when not observed by researchers. One way to address that is to utilize novel modeling methods based on the robot’s sensor data, combined with newer types of interaction evaluation such as ecological momentary assessment (EMA), to recognize behavior modalities. This paper presents such a study of human-specific behavior classification based on data collected through EMA and sensors attached onboard a SAR, which was deployed in user homes. Classification was conducted using generative replay models, which attempt to use encoding/decoding methods to emulate how human dreaming is thought to create perturbations of the same experience in order to learn more efficiently from less data. Both multi-class and binary classification were explored for comparison, using several types of generative replay (variational autoencoders, generative adversarial networks, semi-supervised GANs). The highest-performing binary model showed approximately 79% accuracy (AUC 0.83), though multi-class classification across all modalities only attained 33% accuracy (AUC 0.62, F1 0.25), despite various attempts to improve it. The paper here highlights the strengths and weaknesses of using generative replay for modeling during human–robot interaction in the real world and also suggests a number of research paths for future improvement.
在家中部署社交辅助机器人(SARs),如机器人伴侣宠物,有助于跟踪人类在生活方式随时间变化时的行为和健康相关变化,如 CoVID-19 期间所经历的变化。然而,在人们的日常生活空间部署 SAR 等自主代理时,需要解决的一个基本问题是了解用户在没有被研究人员观察到的情况下是如何与这些机器人互动的。解决这一问题的方法之一是利用基于机器人传感器数据的新型建模方法,并结合生态瞬间评估(EMA)等新型交互评估方法来识别行为模式。本文介绍了基于通过 EMA 和安装在用户家中的合成孔径雷达(SAR)上的传感器收集的数据进行的人类特定行为分类研究。该模型试图使用编码/解码方法来模拟人类做梦时如何对相同体验进行扰动,以便更有效地从更少的数据中学习。为了进行比较,我们使用几种类型的生成式重放(变异自动编码器、生成式对抗网络、半监督 GAN)对多类和二元分类进行了探索。表现最出色的二元模型显示了约 79% 的准确率(AUC 0.83),而所有模式的多类分类仅达到了 33% 的准确率(AUC 0.62,F1 0.25),尽管尝试了各种改进方法。本文强调了在现实世界中使用生成式重放进行人机交互建模的优缺点,并提出了一些未来改进的研究路径。
{"title":"Generative replay for multi-class modeling of human activities via sensor data from in-home robotic companion pets","authors":"Seongcheol Kim, Casey C. Bennett, Zachary Henkel, Jinjae Lee, Cedomir Stanojevic, Kenna Baugus, Cindy L. Bethel, Jennifer A. Piatt, Selma Šabanović","doi":"10.1007/s11370-023-00496-0","DOIUrl":"https://doi.org/10.1007/s11370-023-00496-0","url":null,"abstract":"<p>Deploying socially assistive robots (SARs) at home, such as robotic companion pets, can be useful for tracking behavioral and health-related changes in humans during lifestyle fluctuations over time, like those experienced during CoVID-19. However, a fundamental problem required when deploying autonomous agents such as SARs in people’s everyday living spaces is understanding how users interact with those robots when not observed by researchers. One way to address that is to utilize novel modeling methods based on the robot’s sensor data, combined with newer types of interaction evaluation such as ecological momentary assessment (EMA), to recognize behavior modalities. This paper presents such a study of human-specific behavior classification based on data collected through EMA and sensors attached onboard a SAR, which was deployed in user homes. Classification was conducted using <i>generative replay</i> models, which attempt to use encoding/decoding methods to emulate how human dreaming is thought to create perturbations of the same experience in order to learn more efficiently from less data. Both multi-class and binary classification were explored for comparison, using several types of generative replay (variational autoencoders, generative adversarial networks, semi-supervised GANs). The highest-performing binary model showed approximately 79% accuracy (AUC 0.83), though multi-class classification across all modalities only attained 33% accuracy (AUC 0.62, F1 0.25), despite various attempts to improve it. The paper here highlights the strengths and weaknesses of using generative replay for modeling during human–robot interaction in the real world and also suggests a number of research paths for future improvement.</p>","PeriodicalId":48813,"journal":{"name":"Intelligent Service Robotics","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2023-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138745015","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In order to solve the problem of poor performance of traditional point feature algorithm under low texture and poor illumination, this paper presents a new visual SLAM method based on point–line fusion of line structure constraint. This method first uses an algorithm for homogeneity to process the extracted point features, solving the traditional problem of excessive aggregation and overlap of corner points, which makes the visual front end better able to obtain environmental information. In addition, improved line extraction method algorithm by using the strategy of eliminating the line length makes the line extraction performance twice as efficient as the LSD algorithm, the optical flow tracking algorithm is used to replace the traditional matching algorithm to reduce the running time of the system. In particular, the paper proposes a new constraint on the position of the spatially extracted lines, using the parallelism of 3D lines to correct for degraded lines in the projection process, and adding a new constraint on the line structure to the error function of the whole system, the newly constructed error function is optimized by sliding window, which significantly improves the accuracy and completeness of the whole system in constructing maps. Finally, the performance of the algorithm was tested on a publicly available dataset. The experimental results show that our algorithm performs well in point extraction and matching, the proposed point–line fusion system is better than the popular VINS-mono and PL-VINS algorithms in terms of running time, quality of information obtained, and positioning accuracy.
{"title":"Real-time monocular visual–inertial SLAM with structural constraints of line and point–line fusion","authors":"Shaoshao Wang, Aihua Zhang, Zhiqiang Zhang, Xudong Zhao","doi":"10.1007/s11370-023-00492-4","DOIUrl":"https://doi.org/10.1007/s11370-023-00492-4","url":null,"abstract":"<p>In order to solve the problem of poor performance of traditional point feature algorithm under low texture and poor illumination, this paper presents a new visual SLAM method based on point–line fusion of line structure constraint. This method first uses an algorithm for homogeneity to process the extracted point features, solving the traditional problem of excessive aggregation and overlap of corner points, which makes the visual front end better able to obtain environmental information. In addition, improved line extraction method algorithm by using the strategy of eliminating the line length makes the line extraction performance twice as efficient as the LSD algorithm, the optical flow tracking algorithm is used to replace the traditional matching algorithm to reduce the running time of the system. In particular, the paper proposes a new constraint on the position of the spatially extracted lines, using the parallelism of 3D lines to correct for degraded lines in the projection process, and adding a new constraint on the line structure to the error function of the whole system, the newly constructed error function is optimized by sliding window, which significantly improves the accuracy and completeness of the whole system in constructing maps. Finally, the performance of the algorithm was tested on a publicly available dataset. The experimental results show that our algorithm performs well in point extraction and matching, the proposed point–line fusion system is better than the popular VINS-mono and PL-VINS algorithms in terms of running time, quality of information obtained, and positioning accuracy.</p>","PeriodicalId":48813,"journal":{"name":"Intelligent Service Robotics","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2023-12-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138505076","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-12-02DOI: 10.1007/s11370-023-00493-3
Rogério S. Gonçalves, Talles M. de Carvalho, Pablo B. dos Santos, Frederico C. Souza, Carlos Alberto Gallo, Daniel E. T. Sudbrack, Paulo Victor Trautmann, Bruno C. Clasen, Rafael Z. Homma
To enhance the safety of our airspace, it is essential to implement devices along overhead power lines that effectively reduce the likelihood of collisions involving aircraft, helicopters, balloons, and other airborne objects. Aerial marker balls, which adhere to technical standards concerning their geometry and characteristics, are commonly used for aerial signaling on power transmission systems. Currently, aerial marker balls are installed by technicians either via helicopter or by utilizing ropes to perform the task manually. This process results in significant expenses and exposes the technicians to considerable risk. While robotic methods have been explored, they often present impractical challenges. Despite the advancements in various techniques, difficulties persist in this field. The primary objective of this paper is to design and develop a robotic module that can be attached to a drone, enabling the semi-automated installation of aerial marker balls. The robot model was designed using Computer Aided Design and Computer Aided Engineering software’s, with a subsequent description of the control system. After constructing the drone-robot, it was tested in a simulated environment, proving to be both efficient and cost-effective. This innovative approach improves not only the cost-effectiveness of aerial marker ball installation but also the safety of technicians involved in the process.
{"title":"Drone-robot to install aerial marker balls for power lines","authors":"Rogério S. Gonçalves, Talles M. de Carvalho, Pablo B. dos Santos, Frederico C. Souza, Carlos Alberto Gallo, Daniel E. T. Sudbrack, Paulo Victor Trautmann, Bruno C. Clasen, Rafael Z. Homma","doi":"10.1007/s11370-023-00493-3","DOIUrl":"https://doi.org/10.1007/s11370-023-00493-3","url":null,"abstract":"<p>To enhance the safety of our airspace, it is essential to implement devices along overhead power lines that effectively reduce the likelihood of collisions involving aircraft, helicopters, balloons, and other airborne objects. Aerial marker balls, which adhere to technical standards concerning their geometry and characteristics, are commonly used for aerial signaling on power transmission systems. Currently, aerial marker balls are installed by technicians either via helicopter or by utilizing ropes to perform the task manually. This process results in significant expenses and exposes the technicians to considerable risk. While robotic methods have been explored, they often present impractical challenges. Despite the advancements in various techniques, difficulties persist in this field. The primary objective of this paper is to design and develop a robotic module that can be attached to a drone, enabling the semi-automated installation of aerial marker balls. The robot model was designed using Computer Aided Design and Computer Aided Engineering software’s, with a subsequent description of the control system. After constructing the drone-robot, it was tested in a simulated environment, proving to be both efficient and cost-effective. This innovative approach improves not only the cost-effectiveness of aerial marker ball installation but also the safety of technicians involved in the process. </p>","PeriodicalId":48813,"journal":{"name":"Intelligent Service Robotics","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2023-12-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138543653","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-24DOI: 10.1007/s11370-023-00491-5
Binzhao Xu, Taimur Hassan, Irfan Hussain
Currently, most grasping systems are designed to grasp the static objects only, and grasping dynamic objects has received less attention in the literature. For the traditional manipulation scheme, achieving dynamic grasping requires either a highly precise dynamic model or sophisticated predefined grasping states and gestures, both of which are hard to obtain and tedious to design. In this paper, we develop a novel reinforcement learning (RL)-based dynamic grasping framework with a trajectory prediction module to address these issues. In particular, we divide dynamic grasping into two parts: RL-based grasping strategies learning and trajectory prediction. In the simulation setting, an RL agent is trained to grasp a static object. When this well-trained agent is transferred to the real world, the observation has been augmented with the predicted one from an LSTM-based trajectory prediction module. We validated the proposed method through an experimental setup involving a Baxter manipulator with two finger grippers and an object placed on a moving car. We also evaluated how well RL performs both with and without our intended trajectory prediction. Experiment results demonstrate that our method can grasp the object on different trajectories at various speeds.
{"title":"Improving reinforcement learning based moving object grasping with trajectory prediction","authors":"Binzhao Xu, Taimur Hassan, Irfan Hussain","doi":"10.1007/s11370-023-00491-5","DOIUrl":"https://doi.org/10.1007/s11370-023-00491-5","url":null,"abstract":"<p>Currently, most grasping systems are designed to grasp the static objects only, and grasping dynamic objects has received less attention in the literature. For the traditional manipulation scheme, achieving dynamic grasping requires either a highly precise dynamic model or sophisticated predefined grasping states and gestures, both of which are hard to obtain and tedious to design. In this paper, we develop a novel reinforcement learning (RL)-based dynamic grasping framework with a trajectory prediction module to address these issues. In particular, we divide dynamic grasping into two parts: RL-based grasping strategies learning and trajectory prediction. In the simulation setting, an RL agent is trained to grasp a static object. When this well-trained agent is transferred to the real world, the observation has been augmented with the predicted one from an LSTM-based trajectory prediction module. We validated the proposed method through an experimental setup involving a Baxter manipulator with two finger grippers and an object placed on a moving car. We also evaluated how well RL performs both with and without our intended trajectory prediction. Experiment results demonstrate that our method can grasp the object on different trajectories at various speeds.</p>","PeriodicalId":48813,"journal":{"name":"Intelligent Service Robotics","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2023-11-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138505070","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-11-20DOI: 10.1007/s11370-023-00490-6
Jian Song, Yutian Chen, Xun Liu, Nan Zheng
A rapid and accurate localization scheme is significant for the application of autonomous robots in a prior map. However, this task remains challenging in the real-time requirement due to the complex scan matching. This paper proposes an efficient LiDAR/inertial-based localization method that simplifies the process of scan matching. Firstly, it constructs KD-tree architectures for the prior map in advance and selects sparse point cloud as local map through a novel refined neighborhood search. Then, to ensure the reliability of localization, this method removes the dynamic points in the prior map by the comparison between newly laser scan and the local map. The pose transformation is calculated by the scan matching of edge and planar points from static objects. Finally, this method introduces a uniform motion model to correct the wrong initial guess from incorrect inertial data pre-integration. Three prior maps are collected from typical scenarios through intelligent inspection robot to verify the robustness of proposed method. Experimental results show that the proposed method not only achieves high accuracy of centimeter-level deviation in localization, but takes less than 0.01 s to complete the pose matching when the LiDAR rate is 20 Hz.
{"title":"Efficient LiDAR/inertial-based localization with prior map for autonomous robots","authors":"Jian Song, Yutian Chen, Xun Liu, Nan Zheng","doi":"10.1007/s11370-023-00490-6","DOIUrl":"https://doi.org/10.1007/s11370-023-00490-6","url":null,"abstract":"<p>A rapid and accurate localization scheme is significant for the application of autonomous robots in a prior map. However, this task remains challenging in the real-time requirement due to the complex scan matching. This paper proposes an efficient LiDAR/inertial-based localization method that simplifies the process of scan matching. Firstly, it constructs KD-tree architectures for the prior map in advance and selects sparse point cloud as local map through a novel refined neighborhood search. Then, to ensure the reliability of localization, this method removes the dynamic points in the prior map by the comparison between newly laser scan and the local map. The pose transformation is calculated by the scan matching of edge and planar points from static objects. Finally, this method introduces a uniform motion model to correct the wrong initial guess from incorrect inertial data pre-integration. Three prior maps are collected from typical scenarios through intelligent inspection robot to verify the robustness of proposed method. Experimental results show that the proposed method not only achieves high accuracy of centimeter-level deviation in localization, but takes less than 0.01 s to complete the pose matching when the LiDAR rate is 20 Hz.</p>","PeriodicalId":48813,"journal":{"name":"Intelligent Service Robotics","volume":null,"pages":null},"PeriodicalIF":2.5,"publicationDate":"2023-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138505071","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-10-25DOI: 10.1007/s11370-023-00487-1
Altair Coutinho, Sarang Kim, Hugo Rodrigue
{"title":"Reinforced bidirectional artificial muscles: enhancing force and stability for soft robotics","authors":"Altair Coutinho, Sarang Kim, Hugo Rodrigue","doi":"10.1007/s11370-023-00487-1","DOIUrl":"https://doi.org/10.1007/s11370-023-00487-1","url":null,"abstract":"","PeriodicalId":48813,"journal":{"name":"Intelligent Service Robotics","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-10-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135113696","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-09-28DOI: 10.1007/s11370-023-00486-2
Junwoo Jung, Hyunjin Lee, Chibum Lee
{"title":"Distance estimation with semantic segmentation and edge detection of surround view images","authors":"Junwoo Jung, Hyunjin Lee, Chibum Lee","doi":"10.1007/s11370-023-00486-2","DOIUrl":"https://doi.org/10.1007/s11370-023-00486-2","url":null,"abstract":"","PeriodicalId":48813,"journal":{"name":"Intelligent Service Robotics","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2023-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135386958","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}