首页 > 最新文献

Robotics and Computer-integrated Manufacturing最新文献

英文 中文
GNN-LLM hybrid cognitive architectures for generative task adaptation in multi-human multi-robot collaborative disassembly 多人多机器人协同拆卸中生成任务自适应的GNN-LLM混合认知架构
IF 11.4 1区 计算机科学 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS Pub Date : 2026-04-01 Epub Date: 2025-10-15 DOI: 10.1016/j.rcim.2025.103169
Xiaodong Tong, Ke Li, Jinsong Bao
Traditional human-robot collaboration research has primarily focused on single human-robot dyads, yet faces significant challenges in addressing complex industrial scenarios characterized by concurrent multi-tasking, dynamic disturbances, and heterogeneous role coordination. Transitioning toward multi-human multi-robot collaboration (MHMRC) is crucial for achieving a significant leap in coordinated efficiency and manufacturing flexibility. To address this, we investigate a Hybrid Cognitive Digital Twin (HCDT) framework through generative knowledge-augmented paradigms. Our approach introduces a human-centric cognitive entity to generate task data and knowledge-driven strategies for MHMRC. This work demonstrates that integrating Large Language Models (LLMs) with Graph Neural Networks (GNNs) offers robust capabilities in comprehension, reasoning, ideally meeting MHMRC's requirements for handling unplanned operational variations as well as adapting to dynamic collaborative tasks. Furthermore, we demonstrate that compared to human-engineered precoding strategies, the HCDT-powered MHMRC system autonomously generates collaborative strategies for unscheduled tasks under more complex dynamic conditions and mission scenarios, enabling the execution of situations beyond conventional predefined patterns. The proposed methodology was validated in automotive lithium-ion battery (LIB) disassembly applications. Experimental results demonstrate its adaptability to dynamic collaborative tasks and generalization in generating strategies for unplanned operational variations within dynamic disassembly environments. This approach effectively overcomes various technical challenges to achieve autonomous collaboration in MHMRC systems through knowledge representation, task allocation, and collaborative optimization.
传统的人机协作研究主要集中在单个人机组合上,但在处理具有并发多任务、动态干扰和异构角色协调等特征的复杂工业场景方面面临重大挑战。向多人多机器人协作(MHMRC)过渡对于实现协调效率和制造灵活性的重大飞跃至关重要。为了解决这个问题,我们通过生成知识增强范式研究了混合认知数字孪生(HCDT)框架。我们的方法引入了一个以人为中心的认知实体,为MHMRC生成任务数据和知识驱动策略。这项工作表明,将大型语言模型(llm)与图神经网络(gnn)集成在一起,可以提供强大的理解和推理能力,理想地满足MHMRC处理计划外操作变化的要求,并适应动态协作任务。此外,我们证明,与人为设计的预编码策略相比,hcdt驱动的MHMRC系统在更复杂的动态条件和任务场景下自主生成计划外任务的协作策略,从而能够执行超出常规预定义模式的情况。该方法在汽车锂离子电池(LIB)拆卸应用中得到了验证。实验结果证明了该方法对动态协同任务的适应性,以及在动态拆卸环境下针对计划外操作变化生成策略的通用性。该方法通过知识表示、任务分配和协作优化,有效克服了MHMRC系统中实现自主协作的各种技术挑战。
{"title":"GNN-LLM hybrid cognitive architectures for generative task adaptation in multi-human multi-robot collaborative disassembly","authors":"Xiaodong Tong,&nbsp;Ke Li,&nbsp;Jinsong Bao","doi":"10.1016/j.rcim.2025.103169","DOIUrl":"10.1016/j.rcim.2025.103169","url":null,"abstract":"<div><div>Traditional human-robot collaboration research has primarily focused on single human-robot dyads, yet faces significant challenges in addressing complex industrial scenarios characterized by concurrent multi-tasking, dynamic disturbances, and heterogeneous role coordination. Transitioning toward multi-human multi-robot collaboration (MHMRC) is crucial for achieving a significant leap in coordinated efficiency and manufacturing flexibility. To address this, we investigate a Hybrid Cognitive Digital Twin (HCDT) framework through generative knowledge-augmented paradigms. Our approach introduces a human-centric cognitive entity to generate task data and knowledge-driven strategies for MHMRC. This work demonstrates that integrating Large Language Models (LLMs) with Graph Neural Networks (GNNs) offers robust capabilities in comprehension, reasoning, ideally meeting MHMRC's requirements for handling unplanned operational variations as well as adapting to dynamic collaborative tasks. Furthermore, we demonstrate that compared to human-engineered precoding strategies, the HCDT-powered MHMRC system autonomously generates collaborative strategies for unscheduled tasks under more complex dynamic conditions and mission scenarios, enabling the execution of situations beyond conventional predefined patterns. The proposed methodology was validated in automotive lithium-ion battery (LIB) disassembly applications. Experimental results demonstrate its adaptability to dynamic collaborative tasks and generalization in generating strategies for unplanned operational variations within dynamic disassembly environments. This approach effectively overcomes various technical challenges to achieve autonomous collaboration in MHMRC systems through knowledge representation, task allocation, and collaborative optimization.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103169"},"PeriodicalIF":11.4,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145314951","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
From perception to precision: Vision-based mobile robotic manipulation for assembly screwdriving 从感知到精确:基于视觉的移动机器人操作装配螺丝
IF 11.4 1区 计算机科学 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS Pub Date : 2026-04-01 Epub Date: 2025-09-30 DOI: 10.1016/j.rcim.2025.103148
Aleksandar Stefanov, Miha Zorman, Sebastjan Šlajpah, Janez Podobnik, Matjaž Mihelj, Marko Munih
Flexible manufacturing demands automation that is both precise and adaptable. However, tasks such as screwdriving are typically automated using costly, rigid robotic cells, making this approach impractical for low-volume, high-mix production. As a scalable solution, mobile manipulators offer a flexible alternative, but achieving the required precision for screwdriving remains challenging due to localization uncertainties. This paper addresses these limitations by presenting a vision-guided mobile robotic manipulation system that performs high-precision screwdriving using only monocular RGB imagery. The proposed pipeline integrates stationary and onboard cameras with perception algorithms for object identification and segmentation, pose estimation, and CAD-based screw hole localization, compensating for base misalignment and object placement variability. Experimental validation using ISO 9283 standard’s metrics demonstrates a translational accuracy between 0.21 mm and 0.50 mm across multiple screw positions. Additionally, the system achieves angular estimation errors as low as 0.07°to 0.20°, verifying its capability for sub-degree precision in orientation estimation. In 50 independent experiments involving a total of 400 screw insertions, the system achieved a 100 % success rate, confirming its reliability in practical conditions. These results confirm the feasibility of using RGB-only vision for precision screwdriving and highlight the mobile manipulation system’s scalability for real-world semi-structured manufacturing environments.
柔性制造要求自动化既精确又适应性强。然而,像螺丝刀这样的任务通常是自动化的,使用昂贵的、刚性的机器人单元,这使得这种方法不适合小批量、高混合的生产。作为一种可扩展的解决方案,移动机械手提供了一种灵活的选择,但由于定位的不确定性,实现所需的螺丝刀精度仍然具有挑战性。本文通过提出一种视觉引导的移动机器人操作系统来解决这些限制,该系统仅使用单目RGB图像执行高精度螺丝刀。该管道集成了固定式和机载摄像机,以及用于物体识别和分割、姿态估计和基于cad的螺钉孔定位的感知算法,补偿了基座错位和物体放置的可变性。使用ISO 9283标准度量的实验验证表明,在多个螺钉位置上的平移精度在0.21 mm至0.50 mm之间。此外,该系统的角度估计误差低至0.07°至0.20°,验证了其在定向估计中的亚度精度能力。在50次独立实验中,共进行了400次螺钉置入,成功率达到100%,证实了该系统在实际条件下的可靠性。这些结果证实了仅使用rgb视觉进行精密螺丝刀的可行性,并突出了移动操作系统在现实世界半结构化制造环境中的可扩展性。
{"title":"From perception to precision: Vision-based mobile robotic manipulation for assembly screwdriving","authors":"Aleksandar Stefanov,&nbsp;Miha Zorman,&nbsp;Sebastjan Šlajpah,&nbsp;Janez Podobnik,&nbsp;Matjaž Mihelj,&nbsp;Marko Munih","doi":"10.1016/j.rcim.2025.103148","DOIUrl":"10.1016/j.rcim.2025.103148","url":null,"abstract":"<div><div>Flexible manufacturing demands automation that is both precise and adaptable. However, tasks such as screwdriving are typically automated using costly, rigid robotic cells, making this approach impractical for low-volume, high-mix production. As a scalable solution, mobile manipulators offer a flexible alternative, but achieving the required precision for screwdriving remains challenging due to localization uncertainties. This paper addresses these limitations by presenting a vision-guided mobile robotic manipulation system that performs high-precision screwdriving using only monocular RGB imagery. The proposed pipeline integrates stationary and onboard cameras with perception algorithms for object identification and segmentation, pose estimation, and CAD-based screw hole localization, compensating for base misalignment and object placement variability. Experimental validation using ISO 9283 standard’s metrics demonstrates a translational accuracy between 0.21 mm and 0.50 mm across multiple screw positions. Additionally, the system achieves angular estimation errors as low as 0.07°to 0.20°, verifying its capability for sub-degree precision in orientation estimation. In 50 independent experiments involving a total of 400 screw insertions, the system achieved a 100 % success rate, confirming its reliability in practical conditions. These results confirm the feasibility of using RGB-only vision for precision screwdriving and highlight the mobile manipulation system’s scalability for real-world semi-structured manufacturing environments.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103148"},"PeriodicalIF":11.4,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145220393","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Identification and three-dimensional absorption of time-varying potential chatter during robotic milling 机器人铣削过程中时变电位颤振的识别与三维吸收
IF 11.4 1区 计算机科学 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS Pub Date : 2026-04-01 Epub Date: 2025-11-01 DOI: 10.1016/j.rcim.2025.103173
Jiawei Wu , Rui Fu , Xiaowei Tang , Shihao Xin , Fangyu Peng , Chenyang Wang
Robotic milling constitutes an important component of robotized intelligent manufacturing, gaining increasing popularity for subtractive manufacturing of large components. Extensive efforts have been devoted to the analysis and suppression of robot chatter to enhance milling efficiency and quality. However, the dynamic characteristics of robots are highly pose-dependent, leading to time-varying low-frequency chatter. Meanwhile, the low-frequency chatter is continuously influenced by the action of vibration suppression devices, making it challenging to consistently track and suppress time-varying chatter. To address this, this paper proposes a new concept, the potential chatter mode, to more accurately describe the target mode that requires attention in online chatter suppression. Inspired by the modulation mechanism between modal vibrations and spindle rotation during robotic milling, a potential chatter mode identification framework is developed. By investigating the distribution pattern of vibration spectra under the modulation mechanism, and integrating filtering, demodulation, signal decomposition, and vibration energy evaluation, it achieves the online identification of the time-varying frequency of potential chatter. Furthermore, the potential chatter exhibits a three-dimensional time-varying direction, whereas the existing suppression devices are generally designed to operate in one or two directions. This paper develops a novel three-dimensional orthogonal adaptive vibration absorber (TO-AVA) based on magnetorheological elastomers (MRE). By incorporating a parallel negative stiffness mechanism and parameter design, the TO-AVA can handle the three-dimensional time-varying direction of potential chatter. Validation experiments of robotic milling are conducted, which involves various process parameters and time-varying potential chatter across different directions, frequencies, and states. The results demonstrate that the developed framework can accurately identify time-varying potential chatter and effectively suppress it using the TO-AVA.
机器人铣削加工是机器人智能制造的重要组成部分,在大型零件减法制造中越来越受欢迎。为了提高铣削效率和质量,对机器人颤振进行了大量的分析和抑制。然而,机器人的动态特性高度依赖于姿态,导致时变低频颤振。同时,低频颤振不断受到减振装置作用的影响,为持续跟踪和抑制时变颤振带来了挑战。针对这一问题,本文提出了潜在颤振模式的概念,以更准确地描述在线颤振抑制中需要注意的目标模式。基于机器人铣削过程中模态振动与主轴旋转之间的调制机制,提出了一种潜在颤振模态识别框架。通过研究调制机制下的振动频谱分布规律,将滤波、解调、信号分解、振动能量评价等集成在一起,实现了潜在颤振时变频率的在线辨识。此外,潜在颤振表现出三维时变方向,而现有的抑制装置通常设计为在一个或两个方向上工作。研制了一种基于磁流变弹性体(MRE)的三维正交自适应吸振器。通过并联负刚度机构和参数设计,TO-AVA可以处理三维时变方向的潜在颤振。针对不同工艺参数和不同方向、频率和状态的时变潜在颤振,进行了铣削机器人的验证实验。结果表明,该框架能够准确识别时变潜在颤振,并利用TO-AVA有效抑制时变潜在颤振。
{"title":"Identification and three-dimensional absorption of time-varying potential chatter during robotic milling","authors":"Jiawei Wu ,&nbsp;Rui Fu ,&nbsp;Xiaowei Tang ,&nbsp;Shihao Xin ,&nbsp;Fangyu Peng ,&nbsp;Chenyang Wang","doi":"10.1016/j.rcim.2025.103173","DOIUrl":"10.1016/j.rcim.2025.103173","url":null,"abstract":"<div><div>Robotic milling constitutes an important component of robotized intelligent manufacturing, gaining increasing popularity for subtractive manufacturing of large components. Extensive efforts have been devoted to the analysis and suppression of robot chatter to enhance milling efficiency and quality. However, the dynamic characteristics of robots are highly pose-dependent, leading to time-varying low-frequency chatter. Meanwhile, the low-frequency chatter is continuously influenced by the action of vibration suppression devices, making it challenging to consistently track and suppress time-varying chatter. To address this, this paper proposes a new concept, the potential chatter mode, to more accurately describe the target mode that requires attention in online chatter suppression. Inspired by the modulation mechanism between modal vibrations and spindle rotation during robotic milling, a potential chatter mode identification framework is developed. By investigating the distribution pattern of vibration spectra under the modulation mechanism, and integrating filtering, demodulation, signal decomposition, and vibration energy evaluation, it achieves the online identification of the time-varying frequency of potential chatter. Furthermore, the potential chatter exhibits a three-dimensional time-varying direction, whereas the existing suppression devices are generally designed to operate in one or two directions. This paper develops a novel three-dimensional orthogonal adaptive vibration absorber (TO-AVA) based on magnetorheological elastomers (MRE). By incorporating a parallel negative stiffness mechanism and parameter design, the TO-AVA can handle the three-dimensional time-varying direction of potential chatter. Validation experiments of robotic milling are conducted, which involves various process parameters and time-varying potential chatter across different directions, frequencies, and states. The results demonstrate that the developed framework can accurately identify time-varying potential chatter and effectively suppress it using the TO-AVA.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103173"},"PeriodicalIF":11.4,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145412100","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Accurate localization and girth weld grinding planning for an in-pipe machining robot of thin-walled conical pipe 薄壁锥形管管内加工机器人的精确定位与环焊缝磨削规划
IF 11.4 1区 计算机科学 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS Pub Date : 2026-04-01 Epub Date: 2025-09-16 DOI: 10.1016/j.rcim.2025.103124
Tian Lan , Te Li , Haibo Liu , Shiyu Tian , Kuo Liu , Yongqing Wang
In-pipe robots have been applied to detection, cleaning, welding, grinding, drilling, etc., which realizes the narrow space operation efficiently and economically. However, intelligent operation is still a problem, especially for high-precision machining of inner welds in thin-walled conical pipes, due to the low stiffness and precision of the robot machining system, uncertain robot position, and poor consistency of target characteristics. To address this problem, an intelligent girth weld grinding method of an in-pipe machining robot for thin-walled conical pipes is proposed. An in-pipe machining robot (named IPMR-I) with adaptive motion ability, controllable stiffness, and high machining precision is designed, which benefits by the designs of controllable contact forces and a high-precision three-axis machining mechanism. A weld locating method based on time-series point clouds, combined with an analytical pose model, is used for robot self-localization, obtaining the accurate pose of the robot relative to the weld region. Furthermore, an intelligent machining path planning method is proposed with the abilities of the weld boundary recognition, machining path generation, and optimization, which adaptively realizes the high machining quality and safety facing the welding irregularity (e.g., deformation and misalignment) inside thin-walled conical pipes. Several weld bead grinding experiments were conducted inside thin-walled conical pipes to verify the proposed methods’ validity. The results proved that IPMR-I with the proposed intelligent girth weld grinding method completed autonomous high-quality machining without manual intervention. Without damaging the base material, the maximum residual weld height is 0.18 mm, with the average residual height controlled at approximately 0.08 mm.
管道机器人已应用于检测、清洗、焊接、磨削、钻孔等领域,高效、经济地实现了狭小空间的作业。然而,由于机器人加工系统的刚度和精度较低,机器人位置不确定,目标特性一致性差,智能操作仍然是一个问题,特别是对于薄壁锥形管内焊缝的高精度加工。针对这一问题,提出了一种薄壁锥形管道内加工机器人的智能环焊缝磨削方法。设计了一种具有自适应运动能力、刚度可控、加工精度高的管道加工机器人(ipmr - 1),该机器人得益于接触力可控和高精度三轴加工机构的设计。采用基于时间序列点云的焊缝定位方法,结合解析位姿模型对机器人进行自定位,获得机器人相对于焊缝区域的准确位姿。在此基础上,提出了一种具有焊缝边界识别、加工路径生成和优化能力的智能加工路径规划方法,自适应地实现了薄壁锥形管内部焊接不平整(变形、错位等)的高加工质量和安全性。在薄壁锥形管内进行了焊缝磨削实验,验证了所提方法的有效性。结果表明,采用所提出的环焊缝智能磨削方法的ipmr - 1在没有人工干预的情况下完成了自主高质量加工。在不损坏基材的情况下,最大残焊高度为0.18 mm,平均残焊高度控制在0.08 mm左右。
{"title":"Accurate localization and girth weld grinding planning for an in-pipe machining robot of thin-walled conical pipe","authors":"Tian Lan ,&nbsp;Te Li ,&nbsp;Haibo Liu ,&nbsp;Shiyu Tian ,&nbsp;Kuo Liu ,&nbsp;Yongqing Wang","doi":"10.1016/j.rcim.2025.103124","DOIUrl":"10.1016/j.rcim.2025.103124","url":null,"abstract":"<div><div>In-pipe robots have been applied to detection, cleaning, welding, grinding, drilling, etc., which realizes the narrow space operation efficiently and economically. However, intelligent operation is still a problem, especially for high-precision machining of inner welds in thin-walled conical pipes, due to the low stiffness and precision of the robot machining system, uncertain robot position, and poor consistency of target characteristics. To address this problem, an intelligent girth weld grinding method of an in-pipe machining robot for thin-walled conical pipes is proposed. An in-pipe machining robot (named IPMR-I) with adaptive motion ability, controllable stiffness, and high machining precision is designed, which benefits by the designs of controllable contact forces and a high-precision three-axis machining mechanism. A weld locating method based on time-series point clouds, combined with an analytical pose model, is used for robot self-localization, obtaining the accurate pose of the robot relative to the weld region. Furthermore, an intelligent machining path planning method is proposed with the abilities of the weld boundary recognition, machining path generation, and optimization, which adaptively realizes the high machining quality and safety facing the welding irregularity (e.g., deformation and misalignment) inside thin-walled conical pipes. Several weld bead grinding experiments were conducted inside thin-walled conical pipes to verify the proposed methods’ validity. The results proved that IPMR-I with the proposed intelligent girth weld grinding method completed autonomous high-quality machining without manual intervention. Without damaging the base material, the maximum residual weld height is 0.18 mm, with the average residual height controlled at approximately 0.08 mm.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103124"},"PeriodicalIF":11.4,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145093757","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A generalised system for multi-mobile robot cooperation in smart manufacturing 智能制造中多移动机器人协作的通用系统
IF 11.4 1区 计算机科学 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS Pub Date : 2026-04-01 Epub Date: 2025-09-23 DOI: 10.1016/j.rcim.2025.103139
Tianwei Zhang , Ning Wang , Yiming Yang , Ziya Wang
Since the advent of Industry 4.0, mobile collaborative robot technology has developed rapidly. However, several challenges still hinder the industrial application of mobile collaborative robots. These challenges include human–robot collaboration safety, the complexity of non-standard mobile collaborative task solutions, and the deployment timeliness of heterogeneous robots and mechanical structures. To address these challenges, this paper proposes a general and robust “cloud–edge-terminal-network-intelligence” multi-robot mobile collaboration system. This framework achieves efficient customised solutions by standardising and simplifying robot hardware and software integration. The solution focuses on three key issues: modular design, cloud–edge architecture in advanced manufacturing, and rapid deployment of heterogeneous multi-robots. The paper discusses robot safety and collaborative control issues and provides corresponding technical solutions.
自工业4.0时代到来以来,移动协同机器人技术得到了迅速发展。然而,一些挑战仍然阻碍着移动协作机器人的工业应用。这些挑战包括人机协作的安全性、非标准移动协作任务解决方案的复杂性以及异构机器人和机械结构的部署及时性。针对这些挑战,本文提出了一种通用、鲁棒的“云-边缘-终端-网络智能”多机器人移动协作系统。该框架通过标准化和简化机器人硬件和软件集成来实现高效的定制解决方案。该解决方案侧重于三个关键问题:模块化设计、先进制造中的云边缘架构和异构多机器人的快速部署。讨论了机器人的安全和协同控制问题,并提出了相应的技术解决方案。
{"title":"A generalised system for multi-mobile robot cooperation in smart manufacturing","authors":"Tianwei Zhang ,&nbsp;Ning Wang ,&nbsp;Yiming Yang ,&nbsp;Ziya Wang","doi":"10.1016/j.rcim.2025.103139","DOIUrl":"10.1016/j.rcim.2025.103139","url":null,"abstract":"<div><div>Since the advent of Industry 4.0, mobile collaborative robot technology has developed rapidly. However, several challenges still hinder the industrial application of mobile collaborative robots. These challenges include human–robot collaboration safety, the complexity of non-standard mobile collaborative task solutions, and the deployment timeliness of heterogeneous robots and mechanical structures. To address these challenges, this paper proposes a general and robust “cloud–edge-terminal-network-intelligence” multi-robot mobile collaboration system. This framework achieves efficient customised solutions by standardising and simplifying robot hardware and software integration. The solution focuses on three key issues: modular design, cloud–edge architecture in advanced manufacturing, and rapid deployment of heterogeneous multi-robots. The paper discusses robot safety and collaborative control issues and provides corresponding technical solutions.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103139"},"PeriodicalIF":11.4,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145118870","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Stable grasp generation enabled by part segmentation for real-world robotic applications 通过零件分割实现真实机器人应用的稳定抓取生成
IF 11.4 1区 计算机科学 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS Pub Date : 2026-04-01 Epub Date: 2025-10-28 DOI: 10.1016/j.rcim.2025.103170
Zirui Guo, Xieyuanli Chen, Junkai Ren, Zhiqiang Zheng, Huimin Lu, Ruibin Guo
Robotic manipulation necessitates the capability of advanced perception and grasp generation. Previous approaches for object perception in manipulation mainly rely on original point clouds captured from vision sensors, which exhibit inherent limitations in view perspectives and lack of further analysis of the sensor data. This research introduces implicit representation to facilitate part segmentation from imaging sensors, generating 3D models with structural information that provide grasp generation algorithms with more useful information. Regarding the robotic grasp, prior methods mostly rely on deep learning, which presents satisfactory performance on particular datasets yet raises concerns considering their generalization performance. Instead, this article proposes a novel grasp generation method based on 3D part segmentation, which circumvents the reliance on deep learning techniques. Extensive experimental results show that our approach can proficiently generate approximate part segmentation and high success rate grasps for various objects. By integrating part segmentation with grasp generation, the robot achieves accurate autonomous manipulation as shown in the supplementary video.
机器人操作需要先进的感知和抓取能力。以前的操作对象感知方法主要依赖于从视觉传感器捕获的原始点云,这些方法在视角上存在固有的局限性,并且缺乏对传感器数据的进一步分析。本研究引入隐式表示,以方便从成像传感器中分割零件,生成具有结构信息的三维模型,为抓取生成算法提供更多有用的信息。对于机器人抓取,先前的方法大多依赖于深度学习,它在特定数据集上表现出令人满意的性能,但考虑到其泛化性能,存在一些问题。本文提出了一种新的基于三维零件分割的抓取生成方法,避免了对深度学习技术的依赖。大量的实验结果表明,我们的方法可以熟练地生成近似的零件分割,并且对各种对象的抓取成功率很高。通过将零件分割与抓取生成相结合,机器人实现了精确的自主操作,如补充视频所示。
{"title":"Stable grasp generation enabled by part segmentation for real-world robotic applications","authors":"Zirui Guo,&nbsp;Xieyuanli Chen,&nbsp;Junkai Ren,&nbsp;Zhiqiang Zheng,&nbsp;Huimin Lu,&nbsp;Ruibin Guo","doi":"10.1016/j.rcim.2025.103170","DOIUrl":"10.1016/j.rcim.2025.103170","url":null,"abstract":"<div><div>Robotic manipulation necessitates the capability of advanced perception and grasp generation. Previous approaches for object perception in manipulation mainly rely on original point clouds captured from vision sensors, which exhibit inherent limitations in view perspectives and lack of further analysis of the sensor data. This research introduces implicit representation to facilitate part segmentation from imaging sensors, generating 3D models with structural information that provide grasp generation algorithms with more useful information. Regarding the robotic grasp, prior methods mostly rely on deep learning, which presents satisfactory performance on particular datasets yet raises concerns considering their generalization performance. Instead, this article proposes a novel grasp generation method based on 3D part segmentation, which circumvents the reliance on deep learning techniques. Extensive experimental results show that our approach can proficiently generate approximate part segmentation and high success rate grasps for various objects. By integrating part segmentation with grasp generation, the robot achieves accurate autonomous manipulation as shown in the supplementary video.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103170"},"PeriodicalIF":11.4,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145382962","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Force-sensing-based compliant error compensation control for robotic milling of weak-stiffness thin-walled components 基于力传感的弱刚度薄壁零件机器人铣削柔性误差补偿控制
IF 11.4 1区 计算机科学 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS Pub Date : 2026-04-01 Epub Date: 2025-09-05 DOI: 10.1016/j.rcim.2025.103121
Qunfei Gu , Shun Liu , Sun Jin , Dong Liu
Industrial robots hold considerable potential in the field of milling operations. However, due to their structural characteristics, significant machining errors often occur during the milling process. For the robotic milling, existing research rarely provides direct control strategies for machining error compensation, which limits the further application of industrial robots in high-precision machining tasks. To enhance the robotic milling accuracy, this paper proposes an error compensation control method based on force sensing. First, to predict the relationship between cutting forces and machining parameters, an improved cutting force model is developed by introducing the material removal parameter Sl. Combining the cutting force signals with the robot's position data, the machining error can be predicted. Furthermore, by considering the stiffness characteristics of both the robot and the workpiece, an error compensation control method is proposed. The initial milling trajectory is generated using the robot’s spatial pose and the workpiece model. Based on force sensing and the desired machining accuracy, the cutting parameters are adaptively adjusted. A data-driven adaptive parameter adjustment strategy is further proposed by integrating robot motion data, machining data, and cutting force signals. By adjusting the feed rate in different out-of-tolerance regions, a new compensated milling trajectory is generated to correct machining errors. To validate the effectiveness of the proposed method, robotic milling experiments were conducted on thin-walled light alloy workpieces and feature components. The experimental results demonstrate that the proposed approach significantly reduces machining errors in robotic milling, thereby improving both machining quality and efficiency. These results indicate that the proposed method has strong potential for high-precision robotic milling of complex thin-walled structures.
工业机器人在铣削作业领域具有相当大的潜力。然而,由于其结构特点,在铣削过程中经常出现较大的加工误差。对于机器人铣削,现有的研究很少提供直接的加工误差补偿控制策略,这限制了工业机器人在高精度加工任务中的进一步应用。为了提高机器人铣削精度,提出了一种基于力传感的误差补偿控制方法。首先,为了预测切削力与加工参数之间的关系,引入材料去除参数Sl,建立了改进的切削力模型,将切削力信号与机器人的位置数据相结合,可以预测加工误差。在此基础上,结合机器人和工件的刚度特性,提出了一种误差补偿控制方法。利用机器人的空间位姿和工件模型生成初始铣削轨迹。基于力传感和所需的加工精度,自适应调整切削参数。结合机器人运动数据、加工数据和切削力信号,提出了一种数据驱动的自适应参数调整策略。通过调整不同超差区域的进给速度,生成新的补偿铣削轨迹,从而修正加工误差。为了验证该方法的有效性,对薄壁轻合金工件和特征部件进行了机器人铣削实验。实验结果表明,该方法显著降低了机器人铣削加工误差,提高了加工质量和效率。这些结果表明,该方法在复杂薄壁结构的高精度机器人铣削加工中具有很强的潜力。
{"title":"Force-sensing-based compliant error compensation control for robotic milling of weak-stiffness thin-walled components","authors":"Qunfei Gu ,&nbsp;Shun Liu ,&nbsp;Sun Jin ,&nbsp;Dong Liu","doi":"10.1016/j.rcim.2025.103121","DOIUrl":"10.1016/j.rcim.2025.103121","url":null,"abstract":"<div><div>Industrial robots hold considerable potential in the field of milling operations. However, due to their structural characteristics, significant machining errors often occur during the milling process. For the robotic milling, existing research rarely provides direct control strategies for machining error compensation, which limits the further application of industrial robots in high-precision machining tasks. To enhance the robotic milling accuracy, this paper proposes an error compensation control method based on force sensing. First, to predict the relationship between cutting forces and machining parameters, an improved cutting force model is developed by introducing the material removal parameter S<sub>l</sub>. Combining the cutting force signals with the robot's position data, the machining error can be predicted. Furthermore, by considering the stiffness characteristics of both the robot and the workpiece, an error compensation control method is proposed. The initial milling trajectory is generated using the robot’s spatial pose and the workpiece model. Based on force sensing and the desired machining accuracy, the cutting parameters are adaptively adjusted. A data-driven adaptive parameter adjustment strategy is further proposed by integrating robot motion data, machining data, and cutting force signals. By adjusting the feed rate in different out-of-tolerance regions, a new compensated milling trajectory is generated to correct machining errors. To validate the effectiveness of the proposed method, robotic milling experiments were conducted on thin-walled light alloy workpieces and feature components. The experimental results demonstrate that the proposed approach significantly reduces machining errors in robotic milling, thereby improving both machining quality and efficiency. These results indicate that the proposed method has strong potential for high-precision robotic milling of complex thin-walled structures.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103121"},"PeriodicalIF":11.4,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145005338","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Intelligent support-free additive manufacturing path planning via method library and neural network 基于方法库和神经网络的智能无支撑增材制造路径规划
IF 11.4 1区 计算机科学 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS Pub Date : 2026-04-01 Epub Date: 2025-09-30 DOI: 10.1016/j.rcim.2025.103156
Zhengren Tong , Lai Xu , Xianglong Li , Chen Yang , Qinfeng Wang , Hongyao Shen
Support-free additive manufacturing achieves self-supporting fabrication by adjusting the platform’s posture, effectively reducing material waste and simplifying the post-processing stage. However, the diversity of industrial part geometries requires different approaches to plan the robotic manufacturing paths. Traditional approaches to selecting support-free manufacturing path planning methods rely heavily on expert knowledge. This paper proposes an intelligent path planning system based on a method library. The system utilizes a method library composed of seven approaches to achieve support-free additive manufacturing path planning of various types of parts. The additive manufacturing strategy matching neural network (AMMatcher) is employed to match the optimal path planning method from the method library to a given model and to identify its base surface. AMMatcher can analyze the multi-scale features of the model and leverage a cross-task attention mechanism to propagate classification features into the segmentation task, thereby improving network performance. A newly proposed support-free additive manufacturing model dataset (SFAMDataset) is used to evaluate the performance of AMMatcher and typical samples are validated through fabrication experiments on three different manufacturing platforms. Experimental results demonstrate that AMMatcher effectively identifies suitable manufacturing strategies for various model types and exhibits strong adaptability across different manufacturing platforms.
无支撑增材制造通过调整平台姿态实现自支撑制造,有效减少材料浪费,简化后处理阶段。然而,工业零件几何形状的多样性需要不同的方法来规划机器人制造路径。传统的选择无支持制造路径规划方法严重依赖于专家知识。提出了一种基于方法库的智能路径规划系统。该系统利用由7种方法组成的方法库实现了各类零件的无支撑增材制造路径规划。利用增材制造策略匹配神经网络(AMMatcher)将方法库中的最优路径规划方法与给定模型进行匹配,并识别其基面。AMMatcher可以分析模型的多尺度特征,并利用跨任务注意机制将分类特征传播到分割任务中,从而提高网络性能。利用新提出的无支撑增材制造模型数据集(SFAMDataset)来评估AMMatcher的性能,并通过三种不同制造平台的制造实验对典型样本进行了验证。实验结果表明,AMMatcher能够有效地识别出适合不同模型类型的制造策略,并在不同制造平台上表现出较强的适应性。
{"title":"Intelligent support-free additive manufacturing path planning via method library and neural network","authors":"Zhengren Tong ,&nbsp;Lai Xu ,&nbsp;Xianglong Li ,&nbsp;Chen Yang ,&nbsp;Qinfeng Wang ,&nbsp;Hongyao Shen","doi":"10.1016/j.rcim.2025.103156","DOIUrl":"10.1016/j.rcim.2025.103156","url":null,"abstract":"<div><div>Support-free additive manufacturing achieves self-supporting fabrication by adjusting the platform’s posture, effectively reducing material waste and simplifying the post-processing stage. However, the diversity of industrial part geometries requires different approaches to plan the robotic manufacturing paths. Traditional approaches to selecting support-free manufacturing path planning methods rely heavily on expert knowledge. This paper proposes an intelligent path planning system based on a method library. The system utilizes a method library composed of seven approaches to achieve support-free additive manufacturing path planning of various types of parts. The additive manufacturing strategy matching neural network (AMMatcher) is employed to match the optimal path planning method from the method library to a given model and to identify its base surface. AMMatcher can analyze the multi-scale features of the model and leverage a cross-task attention mechanism to propagate classification features into the segmentation task, thereby improving network performance. A newly proposed support-free additive manufacturing model dataset (SFAMDataset) is used to evaluate the performance of AMMatcher and typical samples are validated through fabrication experiments on three different manufacturing platforms. Experimental results demonstrate that AMMatcher effectively identifies suitable manufacturing strategies for various model types and exhibits strong adaptability across different manufacturing platforms.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103156"},"PeriodicalIF":11.4,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145220967","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Sparse-VMICP: A weak feature point cloud registration algorithm for robotic vision measurement of large complex components 稀疏- vmicp:一种用于大型复杂部件机器人视觉测量的弱特征点云配准算法
IF 11.4 1区 计算机科学 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS Pub Date : 2026-04-01 Epub Date: 2025-11-06 DOI: 10.1016/j.rcim.2025.103175
Xiaozhi Feng , Tao Ding , Hao Wu , Di Li , Ning Jiang , Dahu Zhu
High-precision three-dimensional (3D) measurement of large complex components (LCCs) such as vehicle bodies provides data benchmark for subsequent robotized manufacturing processes. A huge challenge in LCCs measurement is to register the adjacent point clouds with partial overlap, especially when the point cloud geometric features are weak. Despite the existing sparse iterative closest point (Sparse-ICP) registration algorithm uses lp norm to reduce the influence of non-overlapping point clouds during the registration process, however sparse point pairs are prone to fall into local optimum, which causes the registration accuracy to be greatly affected by the initial pose. To overcome the challenging problem, we inherit the advantage of the Sparse-ICP algorithm that the point-to-point distance can suppress tangential slip in the smooth areas. On this basis, we introduce the constraint of point-to-plane distance variance minimization under sparse condition that can suppress the incorrect registration inclination caused by uneven point cloud density, and then present a hybrid algorithm termed as Sparse-VMICP for weak feature point cloud registration. The proposed algorithm aims to enhance the robotic vision measurement accuracy by suppressing registration inclination to adjust the local optimal solution. Robotic vision measurement experiments on two typical LCCs, including high-speed rail body and car bodywork are conducted to verify the superiority of the proposed algorithm. The results demonstrate that the proposed algorithm can effectively reduce the accumulated registration errors in large-scale metrology, compared with other state-of-the-art algorithms, and the stitching measurement accuracy of LCCs can reach 0.012 mm.
大型复杂部件(如车身)的高精度三维测量为后续的机器人制造过程提供了数据基准。在lcc测量中,存在部分重叠的相邻点云的配准是一个巨大的挑战,尤其是在点云几何特征较弱的情况下。尽管现有的稀疏迭代最近点配准算法在配准过程中使用lp范数来减少不重叠点云的影响,但稀疏点对容易陷入局部最优,这使得配准精度受到初始姿态的很大影响。为了克服这一难题,我们继承了稀疏icp算法的优点,即点对点距离可以抑制平滑区域的切向滑动。在此基础上,引入稀疏条件下的点面距离方差最小化约束,抑制了点云密度不均匀导致的配准错误倾向,提出了一种用于弱特征点云配准的稀疏- vmicp混合算法。该算法通过抑制配准倾斜度调整局部最优解来提高机器人视觉测量精度。在高铁车身和汽车车身两种典型lcc上进行了机器人视觉测量实验,验证了该算法的优越性。结果表明,与现有算法相比,该算法能有效降低大尺度计量中累积的配准误差,拼接测量精度可达0.012 mm。
{"title":"Sparse-VMICP: A weak feature point cloud registration algorithm for robotic vision measurement of large complex components","authors":"Xiaozhi Feng ,&nbsp;Tao Ding ,&nbsp;Hao Wu ,&nbsp;Di Li ,&nbsp;Ning Jiang ,&nbsp;Dahu Zhu","doi":"10.1016/j.rcim.2025.103175","DOIUrl":"10.1016/j.rcim.2025.103175","url":null,"abstract":"<div><div>High-precision three-dimensional (3D) measurement of large complex components (LCCs) such as vehicle bodies provides data benchmark for subsequent robotized manufacturing processes. A huge challenge in LCCs measurement is to register the adjacent point clouds with partial overlap, especially when the point cloud geometric features are weak. Despite the existing sparse iterative closest point (Sparse-ICP) registration algorithm uses <em>l<sub>p</sub></em> norm to reduce the influence of non-overlapping point clouds during the registration process, however sparse point pairs are prone to fall into local optimum, which causes the registration accuracy to be greatly affected by the initial pose. To overcome the challenging problem, we inherit the advantage of the Sparse-ICP algorithm that the point-to-point distance can suppress tangential slip in the smooth areas. On this basis, we introduce the constraint of point-to-plane distance variance minimization under sparse condition that can suppress the incorrect registration inclination caused by uneven point cloud density, and then present a hybrid algorithm termed as Sparse-VMICP for weak feature point cloud registration. The proposed algorithm aims to enhance the robotic vision measurement accuracy by suppressing registration inclination to adjust the local optimal solution. Robotic vision measurement experiments on two typical LCCs, including high-speed rail body and car bodywork are conducted to verify the superiority of the proposed algorithm. The results demonstrate that the proposed algorithm can effectively reduce the accumulated registration errors in large-scale metrology, compared with other state-of-the-art algorithms, and the stitching measurement accuracy of LCCs can reach 0.012 mm.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103175"},"PeriodicalIF":11.4,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145447346","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
From insight to autonomous execution: VLM-enhanced embodied agents towards digital twin-assisted human-robot collaborative assembly 从洞察到自主执行:vlm增强的具体代理到数字双辅助人机协作装配
IF 11.4 1区 计算机科学 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS Pub Date : 2026-04-01 Epub Date: 2025-11-05 DOI: 10.1016/j.rcim.2025.103176
Changchun Liu , JiaYe Song , Dunbing Tang , Liping Wang , Haihua Zhu , Qixiang Cai
In recent years, embodied intelligence has emerged as a practicable strategy for accomplishing human-level cognitive abilities, reasoning capacities, and execution capabilities within human-robot collaborative (HRC) assembly scenarios. As the physical instantiation of embodied intelligence, embodied agents remain largely in the exploratory phase; their practical application has yet to mature into a standardized paradigm. A key bottleneck lies in the lack of universally applicable enabling technologies, coupled with a disconnection from physical robot control systems. This deficiency necessitates repetitious training for a variety of functional models when operating in dynamic HRC environments, significantly hindering the ability of embodied agents to acclimate to complicated, dynamically changing collaborative settings. To address this challenge, this study proposes VLM-enhanced embodied agents, specifically tailored to support multimodal cognition, task reasoning, and autonomous execution in digital twin-assisted HRC assembly contexts. The framework is structured through several core steps to realize the full process closed loop from insight to autonomous execution of robots supported by embodied intelligent agents. First, a precise epsilon map relation between the embodied agent and the physical cobot is constructed, thereby enabling the digital characterization and functional capsulation of embodied agents. Building on this agent-based framework, a VLM is developed that integrates domain-specific knowledge with real-time scenario information. This dual-driven design endows the VLM with enhanced perceptual capabilities, allowing it to rapidly recognize and respond to dynamic changes in HRC scenarios. To provide a simulation and deduction engine for embodied reasoning of the assembly task, a digital twin model of the HRC scenario is built to serve as the “embodied brain”. Subsequently, these reasoning results are fed into the VLM serving as invoking parameters for the homologous sub-functional code module. This process facilitates the generation of complete robot motion code, enabling seamless physical execution and thus functioning as the “embodied neuron”. Finally, comparable experiments are conducted in an actual HRC assembly environment. The experimental results demonstrate that the proposed VLM-enhanced embodied agents have competitive advantages in multimodal cognition, task reasoning, and autonomous execution.
近年来,具身智能(embodied intelligence)作为一种可行的策略,在人机协作(HRC)装配场景中实现人类水平的认知能力、推理能力和执行能力。作为具身智能的物理实例,具身代理在很大程度上仍处于探索阶段;它们的实际应用尚未成熟为一个标准化的范例。一个关键的瓶颈在于缺乏普遍适用的使能技术,再加上与物理机器人控制系统的脱节。这一缺陷需要在动态HRC环境中对各种功能模型进行重复训练,这极大地阻碍了具身代理适应复杂、动态变化的协作环境的能力。为了应对这一挑战,本研究提出了vlm增强的具身代理,专门用于支持数字孪生辅助HRC装配环境中的多模态认知、任务推理和自主执行。该框架通过几个核心步骤来构建,以实现由具身智能代理支持的机器人从洞察到自主执行的全过程闭环。首先,构建了具身智能体与物理协作机器人之间精确的epsilon映射关系,从而实现了具身智能体的数字化表征和功能封装。在这个基于代理的框架的基础上,开发了一个集成了特定领域知识和实时场景信息的VLM。这种双驱动设计赋予VLM增强的感知能力,使其能够快速识别和响应HRC场景中的动态变化。为了为装配任务的具身推理提供仿真和推理引擎,构建了HRC场景的数字孪生模型作为“具身大脑”。然后,将这些推理结果作为相应子函数代码模块的调用参数馈送到VLM中。这个过程有利于生成完整的机器人运动代码,实现无缝的物理执行,从而起到“具身神经元”的作用。最后,在实际的HRC装配环境中进行了对比实验。实验结果表明,该方法在多模态认知、任务推理和自主执行等方面具有竞争优势。
{"title":"From insight to autonomous execution: VLM-enhanced embodied agents towards digital twin-assisted human-robot collaborative assembly","authors":"Changchun Liu ,&nbsp;JiaYe Song ,&nbsp;Dunbing Tang ,&nbsp;Liping Wang ,&nbsp;Haihua Zhu ,&nbsp;Qixiang Cai","doi":"10.1016/j.rcim.2025.103176","DOIUrl":"10.1016/j.rcim.2025.103176","url":null,"abstract":"<div><div>In recent years, embodied intelligence has emerged as a practicable strategy for accomplishing human-level cognitive abilities, reasoning capacities, and execution capabilities within human-robot collaborative (HRC) assembly scenarios. As the physical instantiation of embodied intelligence, embodied agents remain largely in the exploratory phase; their practical application has yet to mature into a standardized paradigm. A key bottleneck lies in the lack of universally applicable enabling technologies, coupled with a disconnection from physical robot control systems. This deficiency necessitates repetitious training for a variety of functional models when operating in dynamic HRC environments, significantly hindering the ability of embodied agents to acclimate to complicated, dynamically changing collaborative settings. To address this challenge, this study proposes VLM-enhanced embodied agents, specifically tailored to support multimodal cognition, task reasoning, and autonomous execution in digital twin-assisted HRC assembly contexts. The framework is structured through several core steps to realize the full process closed loop from insight to autonomous execution of robots supported by embodied intelligent agents. First, a precise epsilon map relation between the embodied agent and the physical cobot is constructed, thereby enabling the digital characterization and functional capsulation of embodied agents. Building on this agent-based framework, a VLM is developed that integrates domain-specific knowledge with real-time scenario information. This dual-driven design endows the VLM with enhanced perceptual capabilities, allowing it to rapidly recognize and respond to dynamic changes in HRC scenarios. To provide a simulation and deduction engine for embodied reasoning of the assembly task, a digital twin model of the HRC scenario is built to serve as the “embodied brain”. Subsequently, these reasoning results are fed into the VLM serving as invoking parameters for the homologous sub-functional code module. This process facilitates the generation of complete robot motion code, enabling seamless physical execution and thus functioning as the “embodied neuron”. Finally, comparable experiments are conducted in an actual HRC assembly environment. The experimental results demonstrate that the proposed VLM-enhanced embodied agents have competitive advantages in multimodal cognition, task reasoning, and autonomous execution.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103176"},"PeriodicalIF":11.4,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145441661","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
Robotics and Computer-integrated Manufacturing
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1