Pub Date : 2024-03-05DOI: 10.1017/s0263574724000286
Pengwen Xiong, Yuxuan Huang, Yifan Yin, Yu Zhang, Aiguo Song
Robots with multi-sensors always have a problem of weak pairing among different modals of the collected information produced by multi-sensors, which leads to a bad perception performance during robot interaction. To solve this problem, this paper proposes a Force Vision Sight (FVSight) sensor, which utilizes a distributed flexible tactile sensing array integrated with a vision unit. This innovative approach aims to enhance the overall perceptual capabilities for object recognition. The core idea is using one perceptual layer to trigger both tactile images and force-tactile arrays. It allows the two heterogeneous tactile modal information to be consistent in the temporal and spatial dimensions, thus solving the problem of weak pairing between visual and tactile data. Two experiments are specially designed, namely object classification and slip detection. A dataset containing 27 objects with deep presses and shallow presses is collected for classification, and then 20 slip experiments on three objects are conducted. The determination of slip and stationary state is accurately obtained by covariance operation on the tactile data. The experimental results show the reliability of generated multimodal data and the effectiveness of our proposed FVSight sensor.
{"title":"A novel tactile sensor with multimodal vision and tactile units for multifunctional robot interaction","authors":"Pengwen Xiong, Yuxuan Huang, Yifan Yin, Yu Zhang, Aiguo Song","doi":"10.1017/s0263574724000286","DOIUrl":"https://doi.org/10.1017/s0263574724000286","url":null,"abstract":"<p>Robots with multi-sensors always have a problem of weak pairing among different modals of the collected information produced by multi-sensors, which leads to a bad perception performance during robot interaction. To solve this problem, this paper proposes a Force Vision Sight (FVSight) sensor, which utilizes a distributed flexible tactile sensing array integrated with a vision unit. This innovative approach aims to enhance the overall perceptual capabilities for object recognition. The core idea is using one perceptual layer to trigger both tactile images and force-tactile arrays. It allows the two heterogeneous tactile modal information to be consistent in the temporal and spatial dimensions, thus solving the problem of weak pairing between visual and tactile data. Two experiments are specially designed, namely object classification and slip detection. A dataset containing 27 objects with deep presses and shallow presses is collected for classification, and then 20 slip experiments on three objects are conducted. The determination of slip and stationary state is accurately obtained by covariance operation on the tactile data. The experimental results show the reliability of generated multimodal data and the effectiveness of our proposed FVSight sensor.</p>","PeriodicalId":49593,"journal":{"name":"Robotica","volume":"90 1","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-03-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140034284","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Continuum robot has become a research hotspot due to its excellent dexterity, flexibility and applicability to constrained environments. However, the effective, secure and accurate path planning for the continuum robot remains a challenging issue, for that it is difficult to choose a suitable inverse kinematics solution due to its redundancy in the confined environment. This paper presents a collision-free path planning method based on the improved artificial potential field (APF) for the cable-driven continuum robot, in which the beetle antennae search algorithm is adopted to deal with the optimal problem of APF without the necessary for velocity kinematics. In addition, the local optimum problem of traditional APF is solved by the randomness of the antennae’s direction vector which can make the algorithm easily jump out of local minima. The simulation and experimental results verify the efficiency of the proposed path planning method.
{"title":"Collision-free path planning for cable-driven continuum robot based on improved artificial potential field","authors":"Meng Ding, Xianjie Zheng, Liaoxue Liu, Jian Guo, Yu Guo","doi":"10.1017/s026357472400016x","DOIUrl":"https://doi.org/10.1017/s026357472400016x","url":null,"abstract":"<p>Continuum robot has become a research hotspot due to its excellent dexterity, flexibility and applicability to constrained environments. However, the effective, secure and accurate path planning for the continuum robot remains a challenging issue, for that it is difficult to choose a suitable inverse kinematics solution due to its redundancy in the confined environment. This paper presents a collision-free path planning method based on the improved artificial potential field (APF) for the cable-driven continuum robot, in which the beetle antennae search algorithm is adopted to deal with the optimal problem of APF without the necessary for velocity kinematics. In addition, the local optimum problem of traditional APF is solved by the randomness of the antennae’s direction vector which can make the algorithm easily jump out of local minima. The simulation and experimental results verify the efficiency of the proposed path planning method.</p>","PeriodicalId":49593,"journal":{"name":"Robotica","volume":"24 1","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-03-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140034605","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-04DOI: 10.1017/s0263574724000250
Can He, Lingxiao Meng, Zhirui Sun, Jiankun Wang, Max Q.-H. Meng
Autonomous fabric manipulation is a challenging task due to complex dynamics and potential self-occlusion during fabric handling. An intuitive method of fabric-folding manipulation first involves obtaining a smooth and unfolded fabric configuration before the folding process begins. However, the combination of quasi-static actions like pick & place and dynamic action like fling proves inadequate in effectively unfolding long-sleeved T-shirts with sleeves mostly tucked inside the garment. To address this limitation, this paper introduces an enhanced quasi-static action called pick & drag, specifically designed to handle this type of fabric configuration. Additionally, an efficient dual-arm manipulation system is designed in this paper, which combines quasi-static (including pick & place and pick & drag) and dynamic fling actions to flexibly manipulate fabrics into unfolded and smooth configurations. Subsequently, once it is confirmed that the fabric is sufficiently unfolded and all fabric keypoints are detected, the keypoint-based heuristic folding algorithm is employed for the fabric-folding process. To address the scarcity of publicly available keypoint detection datasets for real fabric, we gathered images of various fabric configurations and types in real scenes to create a comprehensive keypoint dataset for fabric folding. This dataset aims to enhance the success rate of keypoint detection. Moreover, we evaluate the effectiveness of our proposed system in real-world settings, where it consistently and reliably unfolds and folds various types of fabrics, including challenging situations such as long-sleeved T-shirts with most parts of sleeves tucked inside the garment. Specifically, our method achieves a coverage rate of 0.822 and a success rate of 0.88 for long-sleeved T-shirts folding. Supplemental materials and dataset are available on our project webpage at https://sites.google.com/view/fabricfolding.
由于在织物处理过程中存在复杂的动力学和潜在的自闭性,自主织物操作是一项具有挑战性的任务。织物折叠操作的直观方法首先是在折叠过程开始前获得平滑和展开的织物配置。然而,事实证明,将取放等准静态动作和甩动等动态动作结合起来,不足以有效地展开袖子大多塞在衣服里面的长袖 T 恤。为了解决这一局限性,本文引入了一种增强型准静态动作,即 "拾放拖",专门用于处理这种类型的织物配置。此外,本文还设计了一种高效的双臂操纵系统,该系统结合了准静态动作(包括取放和拾取拖拽)和动态甩动动作,可灵活地将织物操纵成展开和平滑的配置。随后,在确认织物已充分展开并检测到所有织物关键点后,就会采用基于关键点的启发式折叠算法进行织物折叠处理。为了解决真实织物公开关键点检测数据集稀缺的问题,我们收集了真实场景中各种织物配置和类型的图像,创建了一个全面的织物折叠关键点数据集。该数据集旨在提高关键点检测的成功率。此外,我们还评估了我们提出的系统在真实世界环境中的有效性,该系统可以稳定可靠地展开和折叠各种类型的织物,包括具有挑战性的情况,例如长袖 T 恤衫的大部分袖子都塞进了衣服里面。具体来说,我们的方法在长袖 T 恤衫的折叠方面达到了 0.822 的覆盖率和 0.88 的成功率。补充材料和数据集可在我们的项目网页 https://sites.google.com/view/fabricfolding 上查阅。
{"title":"FabricFolding: learning efficient fabric folding without expert demonstrations","authors":"Can He, Lingxiao Meng, Zhirui Sun, Jiankun Wang, Max Q.-H. Meng","doi":"10.1017/s0263574724000250","DOIUrl":"https://doi.org/10.1017/s0263574724000250","url":null,"abstract":"Autonomous fabric manipulation is a challenging task due to complex dynamics and potential self-occlusion during fabric handling. An intuitive method of fabric-folding manipulation first involves obtaining a smooth and unfolded fabric configuration before the folding process begins. However, the combination of quasi-static actions like pick & place and dynamic action like fling proves inadequate in effectively unfolding long-sleeved T-shirts with sleeves mostly tucked inside the garment. To address this limitation, this paper introduces an enhanced quasi-static action called pick & drag, specifically designed to handle this type of fabric configuration. Additionally, an efficient dual-arm manipulation system is designed in this paper, which combines quasi-static (including pick & place and pick & drag) and dynamic fling actions to flexibly manipulate fabrics into unfolded and smooth configurations. Subsequently, once it is confirmed that the fabric is sufficiently unfolded and all fabric keypoints are detected, the keypoint-based heuristic folding algorithm is employed for the fabric-folding process. To address the scarcity of publicly available keypoint detection datasets for real fabric, we gathered images of various fabric configurations and types in real scenes to create a comprehensive keypoint dataset for fabric folding. This dataset aims to enhance the success rate of keypoint detection. Moreover, we evaluate the effectiveness of our proposed system in real-world settings, where it consistently and reliably unfolds and folds various types of fabrics, including challenging situations such as long-sleeved T-shirts with most parts of sleeves tucked inside the garment. Specifically, our method achieves a coverage rate of 0.822 and a success rate of 0.88 for long-sleeved T-shirts folding. Supplemental materials and dataset are available on our project webpage at <jats:uri xmlns:xlink=\"http://www.w3.org/1999/xlink\" xlink:href=\"https://sites.google.com/view/fabricfolding\">https://sites.google.com/view/fabricfolding</jats:uri>.","PeriodicalId":49593,"journal":{"name":"Robotica","volume":"65 1","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140034285","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-04DOI: 10.1017/s0263574724000316
Jihua Yin, Xuemei Liu, Youqiang Wang, Yucheng Wang
In the pipeline industry, it is often necessary to monitor cracks and damage in pipelines, or need to clean the inside of the pipeline regularly, or collect adhesive on the inner wall of the pipe, but the pipe is too narrow and difficult for humans to enter, it is necessary to use a pipe machine to complete the work. In this paper, a newly designed screw-driven in-pipe inspection robot (IPIR) is proposed. Compared with common robots, this robot innovatively designs adapting mechanism. The robot can not only adapt to the change of the inner diameter size of the pipeline by using the bionic principle and the deformation characteristics of flexible components but also can pass smoothly in the horizontal/oblique/vertical pipelines and has a certain ability to cross obstacles. In addition, it can transmit images of the inner wall of the pipeline wirelessly for data analysis. Finally, through theoretical analysis and prototype construction, the performance of the robot is verified. The results show that the prototype robot can not only smoothly pass through the acrylic pipe with inner diameter of 120–138 mm but also pass through boss with a height of 3 mm.
{"title":"Design and motion mechanism analysis of screw-driven in-pipe inspection robot based on novel adapting mechanism","authors":"Jihua Yin, Xuemei Liu, Youqiang Wang, Yucheng Wang","doi":"10.1017/s0263574724000316","DOIUrl":"https://doi.org/10.1017/s0263574724000316","url":null,"abstract":"In the pipeline industry, it is often necessary to monitor cracks and damage in pipelines, or need to clean the inside of the pipeline regularly, or collect adhesive on the inner wall of the pipe, but the pipe is too narrow and difficult for humans to enter, it is necessary to use a pipe machine to complete the work. In this paper, a newly designed screw-driven in-pipe inspection robot (IPIR) is proposed. Compared with common robots, this robot innovatively designs adapting mechanism. The robot can not only adapt to the change of the inner diameter size of the pipeline by using the bionic principle and the deformation characteristics of flexible components but also can pass smoothly in the horizontal/oblique/vertical pipelines and has a certain ability to cross obstacles. In addition, it can transmit images of the inner wall of the pipeline wirelessly for data analysis. Finally, through theoretical analysis and prototype construction, the performance of the robot is verified. The results show that the prototype robot can not only smoothly pass through the acrylic pipe with inner diameter of 120–138 mm but also pass through boss with a height of 3 mm.","PeriodicalId":49593,"journal":{"name":"Robotica","volume":"157 1","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140034610","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-03-04DOI: 10.1017/s0263574724000171
Mohamed Amine Ben Abdallah, Imed Khemili, Nizar Aifaoui, Med Amine Laribi
Accurate prediction for mechanisms’ dynamic responses has always been a challenging task for designers. For modeling easiness purposes, mechanisms’ synthesis and optimization have been mostly limited to rigid systems, making consequently the designer unable to vow that the manufactured mechanism satisfies the target responses. To address this limitation, flexible mechanism synthesis is aimed in this work. Two benchmark mechanisms being the core of myriad mechanical devices are of scope, mainly, the flexible slider-crank and the four-bar. In addition to the mechanism dimensions, materials properties have been embedded in the synthesis problem. Two responses are of interest for the slider-crank mechanism, the slider velocity, and the midpoint axial displacement for the flexible connecting rod. Whereas five responses have been compiled for the four-bar mechanism synthesis. A comparative analysis of seven optimization techniques to solve the synthesis problem for both mechanisms has been performed. Subsequently, an executable computer-aided design tool for mechanisms synthesis has been developed under MATLAB®. Numerical outcomes emphasize the limits of a single-response-based synthesis for a flexible mechanism. It has been proven that combining different responses alleviates possible error and fulfill high-accuracy requirement.
{"title":"Computer-aided design tool for typical flexible mechanisms synthesis by means of evolutionary algorithms","authors":"Mohamed Amine Ben Abdallah, Imed Khemili, Nizar Aifaoui, Med Amine Laribi","doi":"10.1017/s0263574724000171","DOIUrl":"https://doi.org/10.1017/s0263574724000171","url":null,"abstract":"Accurate prediction for mechanisms’ dynamic responses has always been a challenging task for designers. For modeling easiness purposes, mechanisms’ synthesis and optimization have been mostly limited to rigid systems, making consequently the designer unable to vow that the manufactured mechanism satisfies the target responses. To address this limitation, flexible mechanism synthesis is aimed in this work. Two benchmark mechanisms being the core of myriad mechanical devices are of scope, mainly, the flexible slider-crank and the four-bar. In addition to the mechanism dimensions, materials properties have been embedded in the synthesis problem. Two responses are of interest for the slider-crank mechanism, the slider velocity, and the midpoint axial displacement for the flexible connecting rod. Whereas five responses have been compiled for the four-bar mechanism synthesis. A comparative analysis of seven optimization techniques to solve the synthesis problem for both mechanisms has been performed. Subsequently, an executable computer-aided design tool for mechanisms synthesis has been developed under MATLAB®. Numerical outcomes emphasize the limits of a single-response-based synthesis for a flexible mechanism. It has been proven that combining different responses alleviates possible error and fulfill high-accuracy requirement.","PeriodicalId":49593,"journal":{"name":"Robotica","volume":"3 1","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140034282","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-26DOI: 10.1017/s0263574724000237
Yating Hu, Qigao Zhou, Zhejun Miao, Hang Yuan, Shuang Liu
The current LiDAR-inertial odometry is prone to cumulative Z-axis error when it runs for a long time. This error can easily lead to the failure to detect the loop-closing in the correct scenario. In this paper, a ground-constrained LiDAR-inertial SLAM is proposed to solve this problem. Reasonable constraints on the ground motion of the mobile robot are incorporated to limit the Z-axis drift error. At the same time, considering the influence of initial positioning error on navigation, a keyframe selection strategy is designed to effectively improve the flatness and accuracy of positioning and the efficiency of loop detection. If GNSS is available, the GNSS factor is added to eliminate the cumulative error of the trajectory. Finally, a large number of experiments are carried out on the self-developed robot platform to verify the effectiveness of the algorithm. The results show that this method can effectively improve location accuracy in outdoor environments, especially in environments of feature degradation and large scale.
目前的激光雷达-惯性里程计在长时间运行时容易产生累积 Z 轴误差。这种误差很容易导致无法在正确的情况下检测到闭环。本文提出了一种地面约束激光雷达-惯性 SLAM 来解决这一问题。本文对移动机器人的地面运动进行了合理的约束,以限制 Z 轴漂移误差。同时,考虑到初始定位误差对导航的影响,设计了一种关键帧选择策略,以有效提高定位的平整度和精度以及环路检测的效率。如果有 GNSS,则加入 GNSS 因子以消除轨迹的累积误差。最后,在自主研发的机器人平台上进行了大量实验,以验证算法的有效性。结果表明,该方法能有效提高室外环境下的定位精度,尤其是在特征退化和大尺度环境下。
{"title":"Outdoor LiDAR-inertial SLAM using ground constraints","authors":"Yating Hu, Qigao Zhou, Zhejun Miao, Hang Yuan, Shuang Liu","doi":"10.1017/s0263574724000237","DOIUrl":"https://doi.org/10.1017/s0263574724000237","url":null,"abstract":"<p>The current LiDAR-inertial odometry is prone to cumulative Z-axis error when it runs for a long time. This error can easily lead to the failure to detect the loop-closing in the correct scenario. In this paper, a ground-constrained LiDAR-inertial SLAM is proposed to solve this problem. Reasonable constraints on the ground motion of the mobile robot are incorporated to limit the Z-axis drift error. At the same time, considering the influence of initial positioning error on navigation, a keyframe selection strategy is designed to effectively improve the flatness and accuracy of positioning and the efficiency of loop detection. If GNSS is available, the GNSS factor is added to eliminate the cumulative error of the trajectory. Finally, a large number of experiments are carried out on the self-developed robot platform to verify the effectiveness of the algorithm. The results show that this method can effectively improve location accuracy in outdoor environments, especially in environments of feature degradation and large scale.</p>","PeriodicalId":49593,"journal":{"name":"Robotica","volume":"12 1","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-02-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139969078","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-26DOI: 10.1017/s0263574724000183
Zhen Feng, Bingxin Xue, Chaoqun Wang, Fengyu Zhou
Safe and socially compliant navigation in a crowded environment is essential for social robots. Numerous research efforts have shown the advantages of deep reinforcement learning techniques in training efficient policies, while most of them ignore fast-moving pedestrians in the crowd. In this paper, we present a novel design of safety measure, named Risk-Area, considering collision theory and motion characteristics of different robots and humans. The geometry of Risk-Area is formed based on the real-time relative positions and velocities of the agents in the environment. Our approach perceives risk in the environment and encourages the robot to take safe and socially compliant navigation behaviors. The proposed method is verified with three existing well-known deep reinforcement learning models in densely populated environments. Experiment results demonstrate that our approach combined with the reinforcement learning techniques can efficiently perceive risk in the environment and navigate the robot with high safety in the crowds with fast-moving pedestrians.
{"title":"Safe and socially compliant robot navigation in crowds with fast-moving pedestrians via deep reinforcement learning","authors":"Zhen Feng, Bingxin Xue, Chaoqun Wang, Fengyu Zhou","doi":"10.1017/s0263574724000183","DOIUrl":"https://doi.org/10.1017/s0263574724000183","url":null,"abstract":"<p>Safe and socially compliant navigation in a crowded environment is essential for social robots. Numerous research efforts have shown the advantages of deep reinforcement learning techniques in training efficient policies, while most of them ignore fast-moving pedestrians in the crowd. In this paper, we present a novel design of safety measure, named Risk-Area, considering collision theory and motion characteristics of different robots and humans. The geometry of Risk-Area is formed based on the real-time relative positions and velocities of the agents in the environment. Our approach perceives risk in the environment and encourages the robot to take safe and socially compliant navigation behaviors. The proposed method is verified with three existing well-known deep reinforcement learning models in densely populated environments. Experiment results demonstrate that our approach combined with the reinforcement learning techniques can efficiently perceive risk in the environment and navigate the robot with high safety in the crowds with fast-moving pedestrians.</p>","PeriodicalId":49593,"journal":{"name":"Robotica","volume":"8 4 1","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-02-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139969082","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The rise in the number of automated robotic kitchens accelerated the need for advanced food handling system, emphasizing food analysis including ingredient classification pose recognition and assembling strategy. Selecting the optimal piece from a pile of similarly shaped food items is a challenge to automated meal assembling system. To address this, we present a constructive assembling algorithm, introducing a unique approach for food pose detection–Fast Image to Pose Detection (FI2PD), and a closed-loop packing strategy. Powered by a convolutional neural network (CNN) and a pose retrieval model, FI2PD is adept at constructing a 6D pose from only RGB images. The method employs a coarse-to-fine approach, leveraging the CNN to pinpoint object orientation and position, alongside a pose retrieval process for target selection and 6D pose derivation. Our closed-loop packing strategy, aided by the Item Arrangement Verifier, ensures precise arrangement and system robustness. Additionally, we introduce our FdIngred328 dataset of nine food categories ranging from fake foods to real foods, and the automatically generated data based on synthetic techniques. The performance of our method for object recognition and pose detection has been demonstrated to achieve a success rate of 97.9%. Impressively, the integration of a closed-loop strategy into our meal-assembly process resulted in a notable success rate of 90%, outperforming the results of systems lacking the closed-loop mechanism.
{"title":"Vision-based food handling system for high-resemblance random food items","authors":"Yadan Zeng, Yee Seng Teoh, Guoniu Zhu, Elvin Toh, I-Ming Chen","doi":"10.1017/s0263574724000122","DOIUrl":"https://doi.org/10.1017/s0263574724000122","url":null,"abstract":"The rise in the number of automated robotic kitchens accelerated the need for advanced food handling system, emphasizing food analysis including ingredient classification pose recognition and assembling strategy. Selecting the optimal piece from a pile of similarly shaped food items is a challenge to automated meal assembling system. To address this, we present a constructive assembling algorithm, introducing a unique approach for food pose detection–Fast Image to Pose Detection (FI2PD), and a closed-loop packing strategy. Powered by a convolutional neural network (CNN) and a pose retrieval model, FI2PD is adept at constructing a 6D pose from only RGB images. The method employs a coarse-to-fine approach, leveraging the CNN to pinpoint object orientation and position, alongside a pose retrieval process for target selection and 6D pose derivation. Our closed-loop packing strategy, aided by the Item Arrangement Verifier, ensures precise arrangement and system robustness. Additionally, we introduce our <jats:italic>FdIngred328</jats:italic> dataset of nine food categories ranging from fake foods to real foods, and the automatically generated data based on synthetic techniques. The performance of our method for object recognition and pose detection has been demonstrated to achieve a success rate of 97.9%. Impressively, the integration of a closed-loop strategy into our meal-assembly process resulted in a notable success rate of 90%, outperforming the results of systems lacking the closed-loop mechanism.","PeriodicalId":49593,"journal":{"name":"Robotica","volume":"5 1","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-02-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139927223","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-02-20DOI: 10.1017/s0263574724000249
Michele Tonan, Alberto Doria, Matteo Bottin, Giulio Rosati
Differentially flat under-actuated robots are characterized by more degrees of freedom (DOF) than actuators: this makes possible the design of lightweight cheap robots with high dexterity. The main issue of such robots is the control of the passive joint, which requires accurate dynamic modeling of the robot. Friction is usually discarded to simplify the models, especially in the case of low-speed trajectories. However, this simplification leads to oscillations of the end-effector about the final position, which are incompatible with fast and accurate motions. This paper focuses on planar $n$ -DOF serial robotic arms with $n-1$ actuated rotational joints plus one final passive rotational joint with stiffness and friction properties. These robots, if properly balanced, are differentially flat. When the non-actuated joint can be considered frictionless, differentially flat robots can be controlled in open loop, calculating the motor torques demanded by point-to-point motions. This paper extends the open-loop control to robots with a passive joint with viscous friction adopting a Laplace transform method. This method can be adopted by exploiting the particular structure of the equations of motion of differentially flat under-actuated robots in which the last equations are linear. Analytical expressions of the motor torques are obtained. The work is enriched by an experimental validation of a $2$ -DOF under-actuated robot and by numerical simulations of the $2$ - and $4$ -DOF robots showing the suppression of unwanted oscillations.
{"title":"Oscillation-free point-to-point motions of planar differentially flat under-actuated robots: a Laplace transform method","authors":"Michele Tonan, Alberto Doria, Matteo Bottin, Giulio Rosati","doi":"10.1017/s0263574724000249","DOIUrl":"https://doi.org/10.1017/s0263574724000249","url":null,"abstract":"Differentially flat under-actuated robots are characterized by more degrees of freedom (DOF) than actuators: this makes possible the design of lightweight cheap robots with high dexterity. The main issue of such robots is the control of the passive joint, which requires accurate dynamic modeling of the robot. Friction is usually discarded to simplify the models, especially in the case of low-speed trajectories. However, this simplification leads to oscillations of the end-effector about the final position, which are incompatible with fast and accurate motions. This paper focuses on planar <jats:inline-formula> <jats:alternatives> <jats:inline-graphic xmlns:xlink=\"http://www.w3.org/1999/xlink\" mime-subtype=\"png\" xlink:href=\"S0263574724000249_inline1.png\" /> <jats:tex-math> $n$ </jats:tex-math> </jats:alternatives> </jats:inline-formula>-DOF serial robotic arms with <jats:inline-formula> <jats:alternatives> <jats:inline-graphic xmlns:xlink=\"http://www.w3.org/1999/xlink\" mime-subtype=\"png\" xlink:href=\"S0263574724000249_inline2.png\" /> <jats:tex-math> $n-1$ </jats:tex-math> </jats:alternatives> </jats:inline-formula> actuated rotational joints plus one final passive rotational joint with stiffness and friction properties. These robots, if properly balanced, are differentially flat. When the non-actuated joint can be considered frictionless, differentially flat robots can be controlled in open loop, calculating the motor torques demanded by point-to-point motions. This paper extends the open-loop control to robots with a passive joint with viscous friction adopting a Laplace transform method. This method can be adopted by exploiting the particular structure of the equations of motion of differentially flat under-actuated robots in which the last equations are linear. Analytical expressions of the motor torques are obtained. The work is enriched by an experimental validation of a <jats:inline-formula> <jats:alternatives> <jats:inline-graphic xmlns:xlink=\"http://www.w3.org/1999/xlink\" mime-subtype=\"png\" xlink:href=\"S0263574724000249_inline3.png\" /> <jats:tex-math> $2$ </jats:tex-math> </jats:alternatives> </jats:inline-formula>-DOF under-actuated robot and by numerical simulations of the <jats:inline-formula> <jats:alternatives> <jats:inline-graphic xmlns:xlink=\"http://www.w3.org/1999/xlink\" mime-subtype=\"png\" xlink:href=\"S0263574724000249_inline4.png\" /> <jats:tex-math> $2$ </jats:tex-math> </jats:alternatives> </jats:inline-formula>- and <jats:inline-formula> <jats:alternatives> <jats:inline-graphic xmlns:xlink=\"http://www.w3.org/1999/xlink\" mime-subtype=\"png\" xlink:href=\"S0263574724000249_inline5.png\" /> <jats:tex-math> $4$ </jats:tex-math> </jats:alternatives> </jats:inline-formula>-DOF robots showing the suppression of unwanted oscillations.","PeriodicalId":49593,"journal":{"name":"Robotica","volume":"47 1","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-02-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139927989","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Real-time and efficient path planning is critical for all robotic systems. In particular, it is of greater importance for industrial robots since the overall planning and execution time directly impact the cycle time and automation economics in production lines. While the problem may not be complex in static environments, classical approaches are inefficient in high-dimensional environments in terms of planning time and optimality. Collision checking poses another challenge in obtaining a real-time solution for path planning in complex environments. To address these issues, we propose an end-to-end learning-based framework viz., Path Planning and Collision checking Network (PPCNet). The PPCNet generates the path by computing waypoints sequentially using two networks: the first network generates a waypoint, and the second one determines whether the waypoint is on a collision-free segment of the path. The end-to-end training process is based on imitation learning that uses data aggregation from the experience of an expert planner to train the two networks, simultaneously. We utilize two approaches for training a network that efficiently approximates the exact geometrical collision checking function. Finally, the PPCNet is evaluated in two different simulation environments and a practical implementation on a robotic arm for a bin-picking application. Compared to the state-of-the-art path-planning methods, our results show significant improvement in performance by greatly reducing the planning time with comparable success rates and path lengths.
{"title":"End-to-end deep learning-based framework for path planning and collision checking: bin-picking application","authors":"Mehran Ghafarian Tamizi, Homayoun Honari, Aleksey Nozdryn-Plotnicki, Homayoun Najjaran","doi":"10.1017/s0263574724000109","DOIUrl":"https://doi.org/10.1017/s0263574724000109","url":null,"abstract":"Real-time and efficient path planning is critical for all robotic systems. In particular, it is of greater importance for industrial robots since the overall planning and execution time directly impact the cycle time and automation economics in production lines. While the problem may not be complex in static environments, classical approaches are inefficient in high-dimensional environments in terms of planning time and optimality. Collision checking poses another challenge in obtaining a real-time solution for path planning in complex environments. To address these issues, we propose an end-to-end learning-based framework viz., Path Planning and Collision checking Network (PPCNet). The PPCNet generates the path by computing waypoints sequentially using two networks: the first network generates a waypoint, and the second one determines whether the waypoint is on a collision-free segment of the path. The end-to-end training process is based on imitation learning that uses data aggregation from the experience of an expert planner to train the two networks, simultaneously. We utilize two approaches for training a network that efficiently approximates the exact geometrical collision checking function. Finally, the PPCNet is evaluated in two different simulation environments and a practical implementation on a robotic arm for a bin-picking application. Compared to the state-of-the-art path-planning methods, our results show significant improvement in performance by greatly reducing the planning time with comparable success rates and path lengths.","PeriodicalId":49593,"journal":{"name":"Robotica","volume":"145 1","pages":""},"PeriodicalIF":2.7,"publicationDate":"2024-02-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139771165","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}