Robotics and Computer-integrated Manufacturing最新文献_第6页

Multimodal action recognition in human–robot collaborative assembly: A contrastive semantic query approach 人机协作装配中的多模态动作识别：一种对比语义查询方法

IF 11.4 1区计算机科学 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS

Robotics and Computer-integrated Manufacturing

Pub Date : 2025-10-24 DOI: 10.1016/j.rcim.2025.103163

Qi Gao , Zhenyu Liu , Mingjie Hou , Guodong Sa , Jianrong Tan

With the increasing demand for flexibility and adaptability in modern manufacturing systems, intelligent perception and recognition of human actions in human-robot collaborative assembly (HRCA) tasks have garnered significant attention. However, accurate action recognition in complex and dynamic environments remains challenging due to challenges in multimodal fusion and semantic understanding. To address these challenges, a semantically-contrastive action recognition network (SCAR) is proposed, which enhances fine-grained modeling and discrimination of assembly actions. SCAR integrates structural motion information from skeleton sequences with semantic and contextual features extracted from RGB images, thereby improving comprehensive scene perception. Furthermore, task-relevant textual descriptions are introduced as semantic priors to guide cross-modal feature learning. A contrastive learning strategy is employed to reinforce semantic alignment and discriminability across modalities, facilitating the learning of task-aware representations. Evaluations on the benchmark action dataset NTU RGB+D and practical HRCA tasks demonstrate that SCAR significantly outperforms mainstream methods in recognition accuracy. The advantage is particularly evident in scenarios involving ambiguous operations and semantically similar assembly tasks. Ablation studies further validate the efficacy of the semantic guidance mechanism and contrastive learning strategy in enhancing modality complementarity and system robustness.

随着现代制造系统对灵活性和适应性要求的不断提高，人机协同装配任务中人的行为的智能感知和识别已经引起了人们的广泛关注。然而，由于多模态融合和语义理解的挑战，在复杂和动态环境中准确的动作识别仍然是一个挑战。为了解决这些问题，提出了一种语义对比动作识别网络（SCAR），该网络增强了装配动作的细粒度建模和识别。SCAR将骨架序列中的结构运动信息与RGB图像中提取的语义和上下文特征相结合，从而提高了场景的综合感知能力。此外，引入与任务相关的文本描述作为语义先验来指导跨模态特征学习。采用对比学习策略来加强语义一致性和跨模态的可判别性，促进任务感知表征的学习。对基准动作数据集NTU RGB+D和实际HRCA任务的评估表明，SCAR在识别精度上明显优于主流方法。在涉及歧义操作和语义相似的组装任务的场景中，这种优势尤为明显。消融研究进一步验证了语义引导机制和对比学习策略在增强模态互补性和系统鲁棒性方面的有效性。

{"title":"Multimodal action recognition in human–robot collaborative assembly: A contrastive semantic query approach","authors":"Qi Gao , Zhenyu Liu , Mingjie Hou , Guodong Sa , Jianrong Tan","doi":"10.1016/j.rcim.2025.103163","DOIUrl":"10.1016/j.rcim.2025.103163","url":null,"abstract":"<div><div>With the increasing demand for flexibility and adaptability in modern manufacturing systems, intelligent perception and recognition of human actions in human-robot collaborative assembly (HRCA) tasks have garnered significant attention. However, accurate action recognition in complex and dynamic environments remains challenging due to challenges in multimodal fusion and semantic understanding. To address these challenges, a semantically-contrastive action recognition network (SCAR) is proposed, which enhances fine-grained modeling and discrimination of assembly actions. SCAR integrates structural motion information from skeleton sequences with semantic and contextual features extracted from RGB images, thereby improving comprehensive scene perception. Furthermore, task-relevant textual descriptions are introduced as semantic priors to guide cross-modal feature learning. A contrastive learning strategy is employed to reinforce semantic alignment and discriminability across modalities, facilitating the learning of task-aware representations. Evaluations on the benchmark action dataset NTU RGB+<em>D</em> and practical HRCA tasks demonstrate that SCAR significantly outperforms mainstream methods in recognition accuracy. The advantage is particularly evident in scenarios involving ambiguous operations and semantically similar assembly tasks. Ablation studies further validate the efficacy of the semantic guidance mechanism and contrastive learning strategy in enhancing modality complementarity and system robustness.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103163"},"PeriodicalIF":11.4,"publicationDate":"2025-10-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145362827","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A gradual disturbance detection model of manufacturing cell: A digital twin driven perspective 制造单元的渐进式干扰检测模型：数字孪生驱动的视角

IF 11.4 1区计算机科学 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS

Robotics and Computer-integrated Manufacturing

Pub Date : 2025-10-23 DOI: 10.1016/j.rcim.2025.103165

Yaguang Zhou , Chao Zhang , Guanghui Zhou , Chong Han , Jiancong Liu , Hongwen Xing , Wei Wang , Ende Ge , Xiaonan Zhang , Asoke K. Nandi

As a modular component of discrete shop-floors, the manufacturing cell offers specific strengths in detecting operation time fluctuations induced by gradual disturbances in the multi-variety, small-batch production mode. Traditional research on abnormal production state detection in shop-floors typically relies on statistical analysis, machine learning, and deep learning methods. However, these methods demonstrate limitations in both comprehensiveness and effectiveness when applied to gradual disturbance detection. Moreover, these studies could solely address the limitations of gradual disturbance detection, without providing insights into how such detection contributes to improvements in the production process. To this end, this study adopts a digital twin driven perspective to not only detect gradual disturbances, but also to associate disturbance detection with bottleneck alleviation and system performance enhancement. Grounded in the synchronization between the physical manufacturing cell in the physical space and its mirrored virtual counterpart in the virtual space, this study models production activities via actual and virtual dynamic graphs in the data space. Within the model space, we jointly employ the convolutional neural network and the graph convolutional network to extract both structured and graph features from production data. The integration across multiple spaces enables digital twin driven of gradual disturbance detection, contributing to bottleneck alleviation and performance enhancement at the system level. This study's comprehensiveness and effectiveness in detecting gradual disturbances are validated on both simulation and actual datasets. Additionally, experiments that inject gradual disturbances into real production scenarios verify that disturbance detection supports both bottleneck alleviation and overall system enhancement.

作为离散车间的模块化组件，制造单元在检测多品种、小批量生产模式中由逐渐干扰引起的操作时间波动方面具有特殊的优势。车间异常生产状态检测的传统研究通常依赖于统计分析、机器学习和深度学习方法。然而，这些方法在应用于渐进式干扰检测时，在全面性和有效性方面都存在局限性。此外，这些研究只能解决渐进式干扰检测的局限性，而不能深入了解这种检测如何有助于改进生产过程。为此，本研究采用数字孪生驱动的视角，不仅检测渐进式干扰，而且将干扰检测与瓶颈缓解和系统性能提升联系起来。基于物理空间中的物理制造单元与其虚拟空间中的镜像虚拟单元之间的同步，本研究通过数据空间中的实际和虚拟动态图形对生产活动进行建模。在模型空间内，我们联合使用卷积神经网络和图卷积网络从生产数据中提取结构化特征和图特征。跨多个空间的集成使数字孪生驱动的逐渐干扰检测，有助于缓解系统层面的瓶颈和性能提高。在模拟和实际数据集上验证了该研究在检测渐变扰动方面的全面性和有效性。此外，将渐进式干扰注入实际生产场景的实验验证了干扰检测支持瓶颈缓解和整体系统增强。

{"title":"A gradual disturbance detection model of manufacturing cell: A digital twin driven perspective","authors":"Yaguang Zhou , Chao Zhang , Guanghui Zhou , Chong Han , Jiancong Liu , Hongwen Xing , Wei Wang , Ende Ge , Xiaonan Zhang , Asoke K. Nandi","doi":"10.1016/j.rcim.2025.103165","DOIUrl":"10.1016/j.rcim.2025.103165","url":null,"abstract":"<div><div>As a modular component of discrete shop-floors, the manufacturing cell offers specific strengths in detecting operation time fluctuations induced by gradual disturbances in the multi-variety, small-batch production mode. Traditional research on abnormal production state detection in shop-floors typically relies on statistical analysis, machine learning, and deep learning methods. However, these methods demonstrate limitations in both comprehensiveness and effectiveness when applied to gradual disturbance detection. Moreover, these studies could solely address the limitations of gradual disturbance detection, without providing insights into how such detection contributes to improvements in the production process. To this end, this study adopts a digital twin driven perspective to not only detect gradual disturbances, but also to associate disturbance detection with bottleneck alleviation and system performance enhancement. Grounded in the synchronization between the physical manufacturing cell in the physical space and its mirrored virtual counterpart in the virtual space, this study models production activities via actual and virtual dynamic graphs in the data space. Within the model space, we jointly employ the convolutional neural network and the graph convolutional network to extract both structured and graph features from production data. The integration across multiple spaces enables digital twin driven of gradual disturbance detection, contributing to bottleneck alleviation and performance enhancement at the system level. This study's comprehensiveness and effectiveness in detecting gradual disturbances are validated on both simulation and actual datasets. Additionally, experiments that inject gradual disturbances into real production scenarios verify that disturbance detection supports both bottleneck alleviation and overall system enhancement.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103165"},"PeriodicalIF":11.4,"publicationDate":"2025-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145362826","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A novel coverage path planning method based on shrink-wrapping technique for autonomous inspection of complex structures using unmanned aerial vehicle 一种基于收缩包裹技术的无人机复杂结构自主检测覆盖路径规划方法

IF 11.4 1区计算机科学 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS

Robotics and Computer-integrated Manufacturing

Pub Date : 2025-10-18 DOI: 10.1016/j.rcim.2025.103149

Burak Kaleci , Gulin Elibol Secil , Sezgin Secil , Zühal Kartal , Metin Ozkan

The inspection of large-scale structures can be challenging, time-consuming, costly, and dangerous. Autonomous robotic systems can provide an effective solution for performing such tasks by overcoming the negative aspects. In this paper, we present a novel coverage path planning method for complete sensor scanning of the outer surface of complex structures using an unmanned aerial vehicle (UAV) with a depth camera. The proposed method introduces a new approach by applying the shrink-wrapping technique to construct a 3D triangular mesh representing the structure's surface boundary. Viewpoints are then generated based on this mesh. Additionally, the triangles within the depth camera's field of view for each viewpoint are determined. The set covering problem (SCP) accepts the set of triangles covered by each viewpoint and reduces the number of viewpoints to decrease the flight distance and time. Finally, the coverage route that includes all the selected viewpoints is defined as the solution to the traveling salesman problem (TSP). We conduct extensive experiments to demonstrate the effectiveness of the proposed method across three different large-scale structures. The results show the validity and effectiveness of the proposed method.

大型结构的检查具有挑战性、耗时、昂贵和危险。自主机器人系统可以为完成这些任务提供有效的解决方案，克服负面影响。本文提出了一种利用深度相机对复杂结构外表面进行全传感器扫描的覆盖路径规划方法。该方法引入了一种新的方法，即利用收缩包绕技术构建一个代表结构表面边界的三维三角形网格。然后根据这个网格生成视点。此外，深度相机的每个视点的视场内的三角形是确定的。集合覆盖问题（SCP）接受每个视点覆盖的三角形集合，并减少视点的数量以减少飞行距离和时间。最后，将包含所有选定视点的覆盖路径定义为旅行商问题（TSP）的解。我们进行了大量的实验来证明所提出的方法在三种不同的大型结构中的有效性。实验结果表明了该方法的有效性。

{"title":"A novel coverage path planning method based on shrink-wrapping technique for autonomous inspection of complex structures using unmanned aerial vehicle","authors":"Burak Kaleci , Gulin Elibol Secil , Sezgin Secil , Zühal Kartal , Metin Ozkan","doi":"10.1016/j.rcim.2025.103149","DOIUrl":"10.1016/j.rcim.2025.103149","url":null,"abstract":"<div><div>The inspection of large-scale structures can be challenging, time-consuming, costly, and dangerous. Autonomous robotic systems can provide an effective solution for performing such tasks by overcoming the negative aspects. In this paper, we present a novel coverage path planning method for complete sensor scanning of the outer surface of complex structures using an unmanned aerial vehicle (UAV) with a depth camera. The proposed method introduces a new approach by applying the shrink-wrapping technique to construct a 3D triangular mesh representing the structure's surface boundary. Viewpoints are then generated based on this mesh. Additionally, the triangles within the depth camera's field of view for each viewpoint are determined. The set covering problem (SCP) accepts the set of triangles covered by each viewpoint and reduces the number of viewpoints to decrease the flight distance and time. Finally, the coverage route that includes all the selected viewpoints is defined as the solution to the traveling salesman problem (TSP). We conduct extensive experiments to demonstrate the effectiveness of the proposed method across three different large-scale structures. The results show the validity and effectiveness of the proposed method.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103149"},"PeriodicalIF":11.4,"publicationDate":"2025-10-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145314952","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

GNN-LLM hybrid cognitive architectures for generative task adaptation in multi-human multi-robot collaborative disassembly 多人多机器人协同拆卸中生成任务自适应的GNN-LLM混合认知架构

IF 11.4 1区计算机科学 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS

Robotics and Computer-integrated Manufacturing

Pub Date : 2025-10-15 DOI: 10.1016/j.rcim.2025.103169

Xiaodong Tong, Ke Li, Jinsong Bao

Traditional human-robot collaboration research has primarily focused on single human-robot dyads, yet faces significant challenges in addressing complex industrial scenarios characterized by concurrent multi-tasking, dynamic disturbances, and heterogeneous role coordination. Transitioning toward multi-human multi-robot collaboration (MHMRC) is crucial for achieving a significant leap in coordinated efficiency and manufacturing flexibility. To address this, we investigate a Hybrid Cognitive Digital Twin (HCDT) framework through generative knowledge-augmented paradigms. Our approach introduces a human-centric cognitive entity to generate task data and knowledge-driven strategies for MHMRC. This work demonstrates that integrating Large Language Models (LLMs) with Graph Neural Networks (GNNs) offers robust capabilities in comprehension, reasoning, ideally meeting MHMRC's requirements for handling unplanned operational variations as well as adapting to dynamic collaborative tasks. Furthermore, we demonstrate that compared to human-engineered precoding strategies, the HCDT-powered MHMRC system autonomously generates collaborative strategies for unscheduled tasks under more complex dynamic conditions and mission scenarios, enabling the execution of situations beyond conventional predefined patterns. The proposed methodology was validated in automotive lithium-ion battery (LIB) disassembly applications. Experimental results demonstrate its adaptability to dynamic collaborative tasks and generalization in generating strategies for unplanned operational variations within dynamic disassembly environments. This approach effectively overcomes various technical challenges to achieve autonomous collaboration in MHMRC systems through knowledge representation, task allocation, and collaborative optimization.

传统的人机协作研究主要集中在单个人机组合上，但在处理具有并发多任务、动态干扰和异构角色协调等特征的复杂工业场景方面面临重大挑战。向多人多机器人协作（MHMRC）过渡对于实现协调效率和制造灵活性的重大飞跃至关重要。为了解决这个问题，我们通过生成知识增强范式研究了混合认知数字孪生（HCDT）框架。我们的方法引入了一个以人为中心的认知实体，为MHMRC生成任务数据和知识驱动策略。这项工作表明，将大型语言模型（llm）与图神经网络（gnn）集成在一起，可以提供强大的理解和推理能力，理想地满足MHMRC处理计划外操作变化的要求，并适应动态协作任务。此外，我们证明，与人为设计的预编码策略相比，hcdt驱动的MHMRC系统在更复杂的动态条件和任务场景下自主生成计划外任务的协作策略，从而能够执行超出常规预定义模式的情况。该方法在汽车锂离子电池（LIB）拆卸应用中得到了验证。实验结果证明了该方法对动态协同任务的适应性，以及在动态拆卸环境下针对计划外操作变化生成策略的通用性。该方法通过知识表示、任务分配和协作优化，有效克服了MHMRC系统中实现自主协作的各种技术挑战。

{"title":"GNN-LLM hybrid cognitive architectures for generative task adaptation in multi-human multi-robot collaborative disassembly","authors":"Xiaodong Tong, Ke Li, Jinsong Bao","doi":"10.1016/j.rcim.2025.103169","DOIUrl":"10.1016/j.rcim.2025.103169","url":null,"abstract":"<div><div>Traditional human-robot collaboration research has primarily focused on single human-robot dyads, yet faces significant challenges in addressing complex industrial scenarios characterized by concurrent multi-tasking, dynamic disturbances, and heterogeneous role coordination. Transitioning toward multi-human multi-robot collaboration (MHMRC) is crucial for achieving a significant leap in coordinated efficiency and manufacturing flexibility. To address this, we investigate a Hybrid Cognitive Digital Twin (HCDT) framework through generative knowledge-augmented paradigms. Our approach introduces a human-centric cognitive entity to generate task data and knowledge-driven strategies for MHMRC. This work demonstrates that integrating Large Language Models (LLMs) with Graph Neural Networks (GNNs) offers robust capabilities in comprehension, reasoning, ideally meeting MHMRC's requirements for handling unplanned operational variations as well as adapting to dynamic collaborative tasks. Furthermore, we demonstrate that compared to human-engineered precoding strategies, the HCDT-powered MHMRC system autonomously generates collaborative strategies for unscheduled tasks under more complex dynamic conditions and mission scenarios, enabling the execution of situations beyond conventional predefined patterns. The proposed methodology was validated in automotive lithium-ion battery (LIB) disassembly applications. Experimental results demonstrate its adaptability to dynamic collaborative tasks and generalization in generating strategies for unplanned operational variations within dynamic disassembly environments. This approach effectively overcomes various technical challenges to achieve autonomous collaboration in MHMRC systems through knowledge representation, task allocation, and collaborative optimization.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103169"},"PeriodicalIF":11.4,"publicationDate":"2025-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145314951","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Human-robot collaborative programming for robotic polishing of impeller using adaptive virtual fixtures and haptic interface 基于自适应虚拟夹具和触觉接口的叶轮机器人抛光人机协同编程

IF 11.4 1区计算机科学 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS

Robotics and Computer-integrated Manufacturing

Pub Date : 2025-10-15 DOI: 10.1016/j.rcim.2025.103166

Zi-Peng Chi , Qing-Hui Wang , Hai-Long Xie , Jian-Long Ni , A.Y.C. Nee , S.K. Ong

Currently planning the toolpath for robotic polishing of aero-engine impellers is still a challenging job due to its narrow and twisted processing channel and prone to various processing interferences. Inspired by the intelligent perception and adaptive decision-making ability of skilled workers, this work proposes a haptic-based human-robot collaborative (HRC) programming interface for robotic polishing of impellers to leverage the experience of skilled operators. With this interface, an HRC programming system is developed by integrating a haptic device, which enables operators to demonstrate a favorable trajectory in a realistic virtual reality (VR) environment by perceiving the polishing force and observing the polishing effect simulated by the system. To enhance the operator's hand-eye coordination ability during HRC programming, an intuitive workspace mapping algorithm between the haptic devices and robots is proposed. In addition, a flexible virtual fixture that can capture the operator's programming intention is proposed, which can adaptively impose appropriate motion and force constraints on the operator’s hand to facilitate interference avoidance and achieve the desired surface quality. The effectiveness and practicability of the proposed approach are validated by toolpath planning and robotic physical polishing experiments of impellers, which shows that the proposed method can reduce the operator’s cognitive load during HRC programming and enhance both programming efficiency and accuracy. Moreover, the method improves both the quality and consistency of polished surfaces since it combines both the advantages of human intelligence and expertise with the high movement accuracy of robots.

目前，航空发动机叶轮机器人抛光加工通道狭窄扭曲，易受各种加工干扰，刀具轨迹规划仍然是一项具有挑战性的工作。受熟练工人的智能感知和自适应决策能力的启发，本研究提出了一种基于触觉的人机协作（HRC）编程接口，用于机器人抛光叶轮，以利用熟练操作员的经验。利用该接口，通过集成触觉设备开发了HRC编程系统，操作人员可以通过感知抛光力和观察系统模拟的抛光效果，在逼真的虚拟现实（VR）环境中演示良好的轨迹。为了提高操作者在HRC编程过程中的手眼协调能力，提出了一种直观的触觉设备与机器人之间的工作空间映射算法。此外，提出了一种能够捕捉操作人员编程意图的柔性虚拟夹具，该夹具可以自适应地对操作人员的手施加适当的运动和力约束，以避免干扰并达到期望的表面质量。通过刀具轨迹规划和叶轮机器人物理抛光实验，验证了所提方法的有效性和实用性，表明所提方法可以减少操作员在HRC编程过程中的认知负荷，提高编程效率和精度。此外，该方法提高了抛光表面的质量和一致性，因为它结合了人类智能和专业知识的优势以及机器人的高运动精度。

{"title":"Human-robot collaborative programming for robotic polishing of impeller using adaptive virtual fixtures and haptic interface","authors":"Zi-Peng Chi , Qing-Hui Wang , Hai-Long Xie , Jian-Long Ni , A.Y.C. Nee , S.K. Ong","doi":"10.1016/j.rcim.2025.103166","DOIUrl":"10.1016/j.rcim.2025.103166","url":null,"abstract":"<div><div>Currently planning the toolpath for robotic polishing of aero-engine impellers is still a challenging job due to its narrow and twisted processing channel and prone to various processing interferences. Inspired by the intelligent perception and adaptive decision-making ability of skilled workers, this work proposes a haptic-based human-robot collaborative (HRC) programming interface for robotic polishing of impellers to leverage the experience of skilled operators. With this interface, an HRC programming system is developed by integrating a haptic device, which enables operators to demonstrate a favorable trajectory in a realistic virtual reality (VR) environment by perceiving the polishing force and observing the polishing effect simulated by the system. To enhance the operator's hand-eye coordination ability during HRC programming, an intuitive workspace mapping algorithm between the haptic devices and robots is proposed. In addition, a flexible virtual fixture that can capture the operator's programming intention is proposed, which can adaptively impose appropriate motion and force constraints on the operator’s hand to facilitate interference avoidance and achieve the desired surface quality. The effectiveness and practicability of the proposed approach are validated by toolpath planning and robotic physical polishing experiments of impellers, which shows that the proposed method can reduce the operator’s cognitive load during HRC programming and enhance both programming efficiency and accuracy. Moreover, the method improves both the quality and consistency of polished surfaces since it combines both the advantages of human intelligence and expertise with the high movement accuracy of robots.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103166"},"PeriodicalIF":11.4,"publicationDate":"2025-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145314953","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A digital twin-driven in-process monitoring system for the ultrasonic vibration-assisted milling 超声振动辅助铣削过程中数字双驱动监控系统

IF 11.4 1区计算机科学 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS

Robotics and Computer-integrated Manufacturing

Pub Date : 2025-10-15 DOI: 10.1016/j.rcim.2025.103168

Xuewei Zhang , Ting Shi , Xianzhen Huang , Tianbiao Yu

The ultrasonic vibration-assisted milling is now widely used in the biomedical, aerospace and electronic manufacturing as its high adaptability and machining efficiency of difficult-to-cut materials. However, the primary challenges including incorrect numerical control (NC) programming code, complicated material removal procedure simulation, reliable in-process machining status monitoring, inevitable interference and collision of cutting tool with workpiece and worktable would limit the application of the ultrasonic vibration-assisted milling. To address these challenges, a digital twin-driven in-process monitoring system is proposed for the ultrasonic vibration-assisted milling process. The fundamental architecture design for the in-process monitoring system is established from the digital twin model for real-time motion control, real-time material removal procedure simulation, in-process physical state analysis, in-process fault diagnosis and historical machining reproduction during the ultrasonic vibration-assisted milling process. The geometric motion control is implemented by converting NC programming code and sensor data into real-time machining trajectories. Meanwhile, the real-time material removal process monitoring is developed with mesh visibility and spatial location relationship for dynamic geometric simulation. More importantly, the fault source is encoded for spindle speed, and the fault diagnosis of the ultrasonic vibration-assisted milling process can be realized by the optimized sparrow search algorithm with back propagation (SSA-BP) neural network. Moreover, the real-time physical state analysis is implemented by the radial basis function (RBF) interpolation and Socket protocols. The function of reproducing historical machining procedure is achieved by the timestamp of historical data in Unity3d script. The proposed monitoring system is validated on the established experiment platform. Correspondingly, the different modules consisting of geometric motion, physical state and data transmission are integrated, moreover, the system client covering client functionality, operational efficiency, and display performance is also tested, in which the efficient and stable operation of the developed in-process monitoring system of ultrasonic vibration-assisted milling can be ensured.

超声振动辅助铣削以其对难切削材料的高适应性和高加工效率，已广泛应用于生物医学、航空航天和电子制造等领域。然而，不正确的数控编程代码、复杂的材料去除过程仿真、可靠的加工状态监测、刀具与工件和工作台不可避免的干涉和碰撞等主要挑战限制了超声振动辅助铣削的应用。为了解决这些挑战，提出了一种用于超声振动辅助铣削过程的数字双驱动过程监控系统。从数字孪生模型出发，建立了超声振动辅助铣削过程中实时运动控制、实时材料去除过程仿真、过程物理状态分析、过程故障诊断和历史加工再现等过程监控系统的基本体系结构设计。几何运动控制是通过将数控编程代码和传感器数据转换为实时加工轨迹来实现的。同时，开发了具有网格可视性和空间位置关系的材料去除过程实时监控，用于动态几何仿真。更重要的是，将故障源编码为主轴转速，利用优化的SSA-BP神经网络麻雀搜索算法实现超声振动辅助铣削过程的故障诊断。通过径向基函数（RBF）插值和Socket协议实现实时物理状态分析。在Unity3d脚本中通过历史数据的时间戳实现历史加工过程的再现功能。在建立的实验平台上对所提出的监测系统进行了验证。相应地，对几何运动、物理状态、数据传输等不同模块进行了集成，并对系统客户端进行了测试，涵盖客户端功能、运行效率、显示性能等方面，保证了所开发的超声振动辅助铣削过程监控系统的高效稳定运行。

{"title":"A digital twin-driven in-process monitoring system for the ultrasonic vibration-assisted milling","authors":"Xuewei Zhang , Ting Shi , Xianzhen Huang , Tianbiao Yu","doi":"10.1016/j.rcim.2025.103168","DOIUrl":"10.1016/j.rcim.2025.103168","url":null,"abstract":"<div><div>The ultrasonic vibration-assisted milling is now widely used in the biomedical, aerospace and electronic manufacturing as its high adaptability and machining efficiency of difficult-to-cut materials. However, the primary challenges including incorrect numerical control (NC) programming code, complicated material removal procedure simulation, reliable in-process machining status monitoring, inevitable interference and collision of cutting tool with workpiece and worktable would limit the application of the ultrasonic vibration-assisted milling. To address these challenges, a digital twin-driven in-process monitoring system is proposed for the ultrasonic vibration-assisted milling process. The fundamental architecture design for the in-process monitoring system is established from the digital twin model for real-time motion control, real-time material removal procedure simulation, in-process physical state analysis, in-process fault diagnosis and historical machining reproduction during the ultrasonic vibration-assisted milling process. The geometric motion control is implemented by converting NC programming code and sensor data into real-time machining trajectories. Meanwhile, the real-time material removal process monitoring is developed with mesh visibility and spatial location relationship for dynamic geometric simulation. More importantly, the fault source is encoded for spindle speed, and the fault diagnosis of the ultrasonic vibration-assisted milling process can be realized by the optimized sparrow search algorithm with back propagation (SSA-BP) neural network. Moreover, the real-time physical state analysis is implemented by the radial basis function (RBF) interpolation and Socket protocols. The function of reproducing historical machining procedure is achieved by the timestamp of historical data in Unity3d script. The proposed monitoring system is validated on the established experiment platform. Correspondingly, the different modules consisting of geometric motion, physical state and data transmission are integrated, moreover, the system client covering client functionality, operational efficiency, and display performance is also tested, in which the efficient and stable operation of the developed in-process monitoring system of ultrasonic vibration-assisted milling can be ensured.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103168"},"PeriodicalIF":11.4,"publicationDate":"2025-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145314954","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Spatiotemporal collaborative digital twin structural health monitoring based on data mechanism fusion 基于数据机制融合的时空协同数字孪生结构健康监测

IF 11.4 1区计算机科学 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS

Robotics and Computer-integrated Manufacturing

Pub Date : 2025-10-13 DOI: 10.1016/j.rcim.2025.103164

Hongjiang Lu , Lilan Liu , Zenggui Gao , Yuyan Yao , Xinjie Cao , Jingwei Tang

Digital twins, by integrating physical entities with virtual models and combining real-time data with physical mechanisms for dynamic interaction and optimization, have become a crucial tool in structural health monitoring. However, existing digital twin models still face limitations in the depth of mechanism and data fusion, as well as in their spatiotemporal collaborative analysis capabilities, which results in poor predictive performance in complex dynamic environments and an inability to fully capture the global state evolution of structures. To address these challenges, this study proposes a spatiotemporal collaborative digital twin (SC-DT) approach for structural health monitoring, which integrates numerical simulation, machine learning, deep learning, surrogate model, and data processing and visualization techniques, enabling real-time monitoring and accurate prediction of structural health status. Using a six-axis robotic arm as an example, the principles and implementation process of the SC-DT method are detailed, and its effectiveness is validated through experiments. Additionally, by comparing the proposed physics-informed hybrid network (PIHN) model with other models, the superiority of the PIHN model in terms of accuracy and effectiveness is demonstrated. Compared to traditional finite element methods, the SC-DT method significantly reduces the time cost of structural performance analysis, achieving instantaneous predictions within 0.1 seconds in the six-axis robotic arm case study, thus providing a novel solution for real-time health monitoring of complex structures.

数字孪生通过将物理实体与虚拟模型相结合，将实时数据与物理机制相结合，进行动态交互和优化，已成为结构健康监测的重要工具。然而，现有的数字孪生模型在机制和数据融合的深度以及时空协同分析能力方面仍然存在局限性，导致在复杂动态环境下的预测性能较差，无法完全捕捉结构的全局状态演变。为了应对这些挑战，本研究提出了一种用于结构健康监测的时空协作数字孪生（SC-DT）方法，该方法集成了数值模拟、机器学习、深度学习、代理模型以及数据处理和可视化技术，能够实时监测和准确预测结构健康状态。以六轴机械臂为例，详细介绍了SC-DT方法的原理和实现过程，并通过实验验证了其有效性。此外，通过将所提出的物理信息混合网络（PIHN）模型与其他模型进行比较，证明了PIHN模型在准确性和有效性方面的优越性。与传统有限元方法相比，SC-DT方法显著降低了结构性能分析的时间成本，在六轴机械臂案例研究中实现了0.1秒内的瞬时预测，为复杂结构的实时健康监测提供了一种新颖的解决方案。

{"title":"Spatiotemporal collaborative digital twin structural health monitoring based on data mechanism fusion","authors":"Hongjiang Lu , Lilan Liu , Zenggui Gao , Yuyan Yao , Xinjie Cao , Jingwei Tang","doi":"10.1016/j.rcim.2025.103164","DOIUrl":"10.1016/j.rcim.2025.103164","url":null,"abstract":"<div><div>Digital twins, by integrating physical entities with virtual models and combining real-time data with physical mechanisms for dynamic interaction and optimization, have become a crucial tool in structural health monitoring. However, existing digital twin models still face limitations in the depth of mechanism and data fusion, as well as in their spatiotemporal collaborative analysis capabilities, which results in poor predictive performance in complex dynamic environments and an inability to fully capture the global state evolution of structures. To address these challenges, this study proposes a spatiotemporal collaborative digital twin (SC-DT) approach for structural health monitoring, which integrates numerical simulation, machine learning, deep learning, surrogate model, and data processing and visualization techniques, enabling real-time monitoring and accurate prediction of structural health status. Using a six-axis robotic arm as an example, the principles and implementation process of the SC-DT method are detailed, and its effectiveness is validated through experiments. Additionally, by comparing the proposed physics-informed hybrid network (PIHN) model with other models, the superiority of the PIHN model in terms of accuracy and effectiveness is demonstrated. Compared to traditional finite element methods, the SC-DT method significantly reduces the time cost of structural performance analysis, achieving instantaneous predictions within 0.1 seconds in the six-axis robotic arm case study, thus providing a novel solution for real-time health monitoring of complex structures.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103164"},"PeriodicalIF":11.4,"publicationDate":"2025-10-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145314956","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Proactive safety reasoning in human-robot collaboration in disassembly through LLM-augmented STPA and FMEA 基于llm增强STPA和FMEA的人机拆卸协作中的主动安全推理

IF 11.4 1区计算机科学 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS

Robotics and Computer-integrated Manufacturing

Pub Date : 2025-10-13 DOI: 10.1016/j.rcim.2025.103162

Morteza Jalali Alenjareghi , Fardin Ghorbani , Samira Keivanpour , Yuvin Adnarain Chinniah , Sabrina Jocelyn

Disassembly tasks in human–robot collaboration (HRC) environments present safety challenges due to hazardous materials, control system variability, and physically demanding operator tasks. To address these challenges, we propose an AI-augmented risk assessment framework integrating System-Theoretic Process Analysis (STPA) and Failure Mode and Effects Analysis (FMEA). This framework is implemented in four configurations: Term Frequency–Inverse Document Frequency (TF-IDF), Fine-tuned Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and RAG with a structured Knowledge Graph (KG) built from safety standards. The system supports real-time, standards-compliant safety reasoning by generating interpretable, context-specific recommendations. We evaluate these configurations across GPT-3.5 TURBO, GPT-4o, GPT-4.1, and open-source LLMs Qwen2.5 (3B) and Ministral (3B). Among all, RAG+KG with GPT-4.1 achieved the highest results across language-based metrics (BLEU: 68.3, ROUGE-L: 72.0, Semantic Similarity: 81.1, BERTScore (F1): 90.0) and safety-specific metrics (Hazard Recall: 92, Compliance Precision: 97, Safety Violation Rate: zero). Six safety-oriented metrics were introduced to assess compliance, hazard coverage, interpretability, and robustness. A case study on electrical vehicle (EV) battery module disassembly demonstrated the system’s effectiveness in identifying unsafe control actions, tracing failure modes, and recommending targeted mitigation strategies for mechanical, electrical, and chemical hazards, and ergonomic considerations. This framework offers a scalable, explainable approach to real-time safety analysis, advancing AI-enabled risk assessment in dynamic HRC disassembly tasks and supporting the vision of human-centered Industry 5.0 manufacturing.

在人机协作（HRC）环境中，由于危险材料、控制系统的可变性和对操作者体力要求高的任务，拆卸任务带来了安全挑战。为了解决这些挑战，我们提出了一个集成系统理论过程分析（STPA）和失效模式和影响分析（FMEA）的人工智能增强风险评估框架。该框架以四种配置实现：术语频率-逆文档频率（TF-IDF）、微调大型语言模型（llm）、检索-增强生成（RAG）和基于安全标准构建的结构化知识图（KG）的RAG。该系统通过生成可解释的、特定于上下文的建议来支持实时的、符合标准的安全推理。我们在GPT-3.5 TURBO、gpt - 40、GPT-4.1和开源LLMs Qwen2.5 （3B）和Ministral （3B）中评估了这些配置。其中，GPT-4.1的RAG+KG在基于语言的指标（BLEU: 68.3, ROUGE-L: 72.0，语义相似度：81.1,BERTScore (F1): 90.0）和安全特定指标（Hazard Recall: 92, Compliance Precision: 97, Safety Violation Rate: 0）上取得了最高的结果。引入了六个面向安全的度量来评估合规性、危害覆盖范围、可解释性和健壮性。电动汽车（EV）电池模块拆卸的案例研究表明，该系统在识别不安全控制行为、追踪故障模式、针对机械、电气和化学危害提出有针对性的缓解策略以及人体工程学考虑方面是有效的。该框架提供了一种可扩展、可解释的实时安全分析方法，推进了人工智能在动态HRC拆卸任务中的风险评估，并支持以人为中心的工业5.0制造愿景。

{"title":"Proactive safety reasoning in human-robot collaboration in disassembly through LLM-augmented STPA and FMEA","authors":"Morteza Jalali Alenjareghi , Fardin Ghorbani , Samira Keivanpour , Yuvin Adnarain Chinniah , Sabrina Jocelyn","doi":"10.1016/j.rcim.2025.103162","DOIUrl":"10.1016/j.rcim.2025.103162","url":null,"abstract":"<div><div>Disassembly tasks in human–robot collaboration (HRC) environments present safety challenges due to hazardous materials, control system variability, and physically demanding operator tasks. To address these challenges, we propose an AI-augmented risk assessment framework integrating System-Theoretic Process Analysis (STPA) and Failure Mode and Effects Analysis (FMEA). This framework is implemented in four configurations: Term Frequency–Inverse Document Frequency (TF-IDF), Fine-tuned Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and RAG with a structured Knowledge Graph (KG) built from safety standards. The system supports real-time, standards-compliant safety reasoning by generating interpretable, context-specific recommendations. We evaluate these configurations across GPT-3.5 TURBO, GPT-4o, GPT-4.1, and open-source LLMs Qwen2.5 (3B) and Ministral (3B). Among all, RAG+KG with GPT-4.1 achieved the highest results across language-based metrics (BLEU: 68.3, ROUGE-L: 72.0, Semantic Similarity: 81.1, BERTScore (F1): 90.0) and safety-specific metrics (Hazard Recall: 92, Compliance Precision: 97, Safety Violation Rate: zero). Six safety-oriented metrics were introduced to assess compliance, hazard coverage, interpretability, and robustness. A case study on electrical vehicle (EV) battery module disassembly demonstrated the system’s effectiveness in identifying unsafe control actions, tracing failure modes, and recommending targeted mitigation strategies for mechanical, electrical, and chemical hazards, and ergonomic considerations. This framework offers a scalable, explainable approach to real-time safety analysis, advancing AI-enabled risk assessment in dynamic HRC disassembly tasks and supporting the vision of human-centered Industry 5.0 manufacturing.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103162"},"PeriodicalIF":11.4,"publicationDate":"2025-10-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145314955","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Perception-decision-execution coordination mechanism driven dynamic autonomous collaboration method for human-like collaborative robot based on multimodal large language model 基于多模态大语言模型的仿人协作机器人感知-决策-执行协调机制驱动的动态自主协作方法

IF 11.4 1区计算机科学 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS

Robotics and Computer-integrated Manufacturing

Pub Date : 2025-10-11 DOI: 10.1016/j.rcim.2025.103167

Jianpeng Chen , Sihan Huang , Xiaowen Wang , Pengfei Wang , Jiahao Zhu , Zhe Xu , Guoxin Wang , Yan Yan , Lihui Wang

With the advent of Industry 5.0, human-centric smart manufacturing is becoming a new paradigm for industrial transformation. Human-robot collaboration (HRC) is the hot topic of human-centric smart manufacturing. The emergence of large language model (LLM) provides significant opportunity for collaborative robot to promote the autonomous collaboration ability, which brings HRC into new era driven by embodied intelligence and more powerful robot. Therefore, a dynamic autonomous collaboration method inspired from looking-thinking-doing chain of human operators is proposed for human-like collaborative robot (HLCobot) in human-centric smart manufacturing based on multimodal large language model (MLLM), where perception-decision-execution coordination mechanism is constructed to appropriately distribute the abilities of MLLM in the dynamic operation chain of HRC. Firstly, a brain-inspired architecture with the integration of perception hub, decision hub, and execution hub is designed for dynamic autonomous collaboration. Secondly, the abilities of perception, decision, execution of HLCobot are realized by integrating MLLM, where the HLCobot can actively recognize the dynamic changes of HRC scenario by mimicking human operator and execute the correct motions to complete the necessary collaborative task autonomously. Additionally, a coordination mechanism among the agents of perception, decision, and execution is put forward to proceed the collaborative task smoothly. Finally, a case study of engine assembly is provided to demonstrate the effectiveness of the proposed method.

随着工业5.0的到来，以人为中心的智能制造正在成为产业转型的新范式。人机协作（Human-robot collaboration， HRC）是以人为中心的智能制造的热点。大语言模型（large language model， LLM）的出现为协作机器人自主协作能力的提升提供了重要契机，使HRC进入了由具身智能和更强大的机器人驱动的新时代。因此，基于多模态大语言模型（multimodal large language model， MLLM），针对以人为中心的智能制造中的类人协作机器人（HLCobot），提出了一种受人类操作者“看-想-做”链启发的动态自主协作方法，构建感知-决策-执行协调机制，将MLLM的能力在HRC的动态操作链中合理分配。首先，设计了一种集成感知中心、决策中心和执行中心的基于大脑的动态自主协作架构；其次，通过整合MLLM实现HLCobot的感知、决策、执行能力，HLCobot可以通过模仿人类操作员，主动识别HRC场景的动态变化，并执行正确的动作，自主完成必要的协同任务。在此基础上，提出了感知、决策和执行agent之间的协调机制，以保证协同任务的顺利进行。最后，以发动机总成为例，验证了该方法的有效性。

{"title":"Perception-decision-execution coordination mechanism driven dynamic autonomous collaboration method for human-like collaborative robot based on multimodal large language model","authors":"Jianpeng Chen , Sihan Huang , Xiaowen Wang , Pengfei Wang , Jiahao Zhu , Zhe Xu , Guoxin Wang , Yan Yan , Lihui Wang","doi":"10.1016/j.rcim.2025.103167","DOIUrl":"10.1016/j.rcim.2025.103167","url":null,"abstract":"<div><div>With the advent of Industry 5.0, human-centric smart manufacturing is becoming a new paradigm for industrial transformation. Human-robot collaboration (HRC) is the hot topic of human-centric smart manufacturing. The emergence of large language model (LLM) provides significant opportunity for collaborative robot to promote the autonomous collaboration ability, which brings HRC into new era driven by embodied intelligence and more powerful robot. Therefore, a dynamic autonomous collaboration method inspired from looking-thinking-doing chain of human operators is proposed for human-like collaborative robot (HLCobot) in human-centric smart manufacturing based on multimodal large language model (MLLM), where perception-decision-execution coordination mechanism is constructed to appropriately distribute the abilities of MLLM in the dynamic operation chain of HRC. Firstly, a brain-inspired architecture with the integration of perception hub, decision hub, and execution hub is designed for dynamic autonomous collaboration. Secondly, the abilities of perception, decision, execution of HLCobot are realized by integrating MLLM, where the HLCobot can actively recognize the dynamic changes of HRC scenario by mimicking human operator and execute the correct motions to complete the necessary collaborative task autonomously. Additionally, a coordination mechanism among the agents of perception, decision, and execution is put forward to proceed the collaborative task smoothly. Finally, a case study of engine assembly is provided to demonstrate the effectiveness of the proposed method.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103167"},"PeriodicalIF":11.4,"publicationDate":"2025-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145262007","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A hierarchical spatial–aware algorithm with efficient reinforcement learning for human–robot task planning and allocation in production 面向生产中人机任务规划与分配的分层空间感知高效强化学习算法

IF 11.4 1区计算机科学 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS

Robotics and Computer-integrated Manufacturing

Pub Date : 2025-10-10 DOI: 10.1016/j.rcim.2025.103159

Jintao Xue, Xiao Li, Nianmin Zhang

In advanced manufacturing systems, humans and robots collaborate to conduct the production process. Effective task planning and allocation (TPA) is crucial for achieving high production efficiency, yet it remains challenging in complex and dynamic manufacturing environments. The dynamic nature of humans and robots, particularly the need to consider spatial information (e.g., humans’ real-time position and the distance they need to move to complete a task), substantially complicates TPA. To address the above challenges, we decompose production tasks into manageable subtasks. We then implement a real-time hierarchical human–robot TPA algorithm, including a high-level agent for task planning and a low-level agent for task allocation. For the high-level agent, we propose an efficient buffer-based deep Q-learning method (EBQ), which reduces training time and enhances performance in production problems with long-term and sparse reward challenges. For the low-level agent, a path planning-based spatially aware method (SAP) is designed to allocate tasks to the appropriate human–robot resources, thereby achieving the corresponding sequential subtasks. We conducted experiments on a complex real-time production process in a 3D simulator. The results demonstrate that our proposed EBQ&SAP method effectively addresses human–robot TPA problems in complex and dynamic production processes.

在先进的制造系统中，人类和机器人合作进行生产过程。有效的任务规划和分配（TPA）是实现高生产效率的关键，但在复杂和动态的制造环境中仍然具有挑战性。人类和机器人的动态特性，特别是需要考虑空间信息（例如，人类的实时位置和他们完成任务所需移动的距离），大大复杂化了TPA。为了解决上述挑战，我们将生产任务分解为可管理的子任务。然后，我们实现了一个实时分层人-机器人TPA算法，包括一个用于任务规划的高级代理和一个用于任务分配的低级代理。对于高级智能体，我们提出了一种高效的基于缓冲区的深度q学习方法（EBQ），该方法减少了训练时间，并提高了具有长期和稀疏奖励挑战的生产问题的性能。对于底层智能体，设计了一种基于路径规划的空间感知方法（SAP），将任务分配到适当的人机资源中，从而实现相应的顺序子任务。我们在3D模拟器上对一个复杂的实时生产过程进行了实验。结果表明，我们提出的EBQ&；SAP方法有效地解决了复杂动态生产过程中的人机TPA问题。

{"title":"A hierarchical spatial–aware algorithm with efficient reinforcement learning for human–robot task planning and allocation in production","authors":"Jintao Xue, Xiao Li, Nianmin Zhang","doi":"10.1016/j.rcim.2025.103159","DOIUrl":"10.1016/j.rcim.2025.103159","url":null,"abstract":"<div><div>In advanced manufacturing systems, humans and robots collaborate to conduct the production process. Effective task planning and allocation (TPA) is crucial for achieving high production efficiency, yet it remains challenging in complex and dynamic manufacturing environments. The dynamic nature of humans and robots, particularly the need to consider spatial information (e.g., humans’ real-time position and the distance they need to move to complete a task), substantially complicates TPA. To address the above challenges, we decompose production tasks into manageable subtasks. We then implement a real-time hierarchical human–robot TPA algorithm, including a high-level agent for task planning and a low-level agent for task allocation. For the high-level agent, we propose an efficient buffer-based deep Q-learning method (EBQ), which reduces training time and enhances performance in production problems with long-term and sparse reward challenges. For the low-level agent, a path planning-based spatially aware method (SAP) is designed to allocate tasks to the appropriate human–robot resources, thereby achieving the corresponding sequential subtasks. We conducted experiments on a complex real-time production process in a 3D simulator. The results demonstrate that our proposed EBQ&SAP method effectively addresses human–robot TPA problems in complex and dynamic production processes.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103159"},"PeriodicalIF":11.4,"publicationDate":"2025-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145261988","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0