The inspection of large-scale structures can be challenging, time-consuming, costly, and dangerous. Autonomous robotic systems can provide an effective solution for performing such tasks by overcoming the negative aspects. In this paper, we present a novel coverage path planning method for complete sensor scanning of the outer surface of complex structures using an unmanned aerial vehicle (UAV) with a depth camera. The proposed method introduces a new approach by applying the shrink-wrapping technique to construct a 3D triangular mesh representing the structure's surface boundary. Viewpoints are then generated based on this mesh. Additionally, the triangles within the depth camera's field of view for each viewpoint are determined. The set covering problem (SCP) accepts the set of triangles covered by each viewpoint and reduces the number of viewpoints to decrease the flight distance and time. Finally, the coverage route that includes all the selected viewpoints is defined as the solution to the traveling salesman problem (TSP). We conduct extensive experiments to demonstrate the effectiveness of the proposed method across three different large-scale structures. The results show the validity and effectiveness of the proposed method.
{"title":"A novel coverage path planning method based on shrink-wrapping technique for autonomous inspection of complex structures using unmanned aerial vehicle","authors":"Burak Kaleci , Gulin Elibol Secil , Sezgin Secil , Zühal Kartal , Metin Ozkan","doi":"10.1016/j.rcim.2025.103149","DOIUrl":"10.1016/j.rcim.2025.103149","url":null,"abstract":"<div><div>The inspection of large-scale structures can be challenging, time-consuming, costly, and dangerous. Autonomous robotic systems can provide an effective solution for performing such tasks by overcoming the negative aspects. In this paper, we present a novel coverage path planning method for complete sensor scanning of the outer surface of complex structures using an unmanned aerial vehicle (UAV) with a depth camera. The proposed method introduces a new approach by applying the shrink-wrapping technique to construct a 3D triangular mesh representing the structure's surface boundary. Viewpoints are then generated based on this mesh. Additionally, the triangles within the depth camera's field of view for each viewpoint are determined. The set covering problem (SCP) accepts the set of triangles covered by each viewpoint and reduces the number of viewpoints to decrease the flight distance and time. Finally, the coverage route that includes all the selected viewpoints is defined as the solution to the traveling salesman problem (TSP). We conduct extensive experiments to demonstrate the effectiveness of the proposed method across three different large-scale structures. The results show the validity and effectiveness of the proposed method.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103149"},"PeriodicalIF":11.4,"publicationDate":"2025-10-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145314952","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-10-15DOI: 10.1016/j.rcim.2025.103169
Xiaodong Tong, Ke Li, Jinsong Bao
Traditional human-robot collaboration research has primarily focused on single human-robot dyads, yet faces significant challenges in addressing complex industrial scenarios characterized by concurrent multi-tasking, dynamic disturbances, and heterogeneous role coordination. Transitioning toward multi-human multi-robot collaboration (MHMRC) is crucial for achieving a significant leap in coordinated efficiency and manufacturing flexibility. To address this, we investigate a Hybrid Cognitive Digital Twin (HCDT) framework through generative knowledge-augmented paradigms. Our approach introduces a human-centric cognitive entity to generate task data and knowledge-driven strategies for MHMRC. This work demonstrates that integrating Large Language Models (LLMs) with Graph Neural Networks (GNNs) offers robust capabilities in comprehension, reasoning, ideally meeting MHMRC's requirements for handling unplanned operational variations as well as adapting to dynamic collaborative tasks. Furthermore, we demonstrate that compared to human-engineered precoding strategies, the HCDT-powered MHMRC system autonomously generates collaborative strategies for unscheduled tasks under more complex dynamic conditions and mission scenarios, enabling the execution of situations beyond conventional predefined patterns. The proposed methodology was validated in automotive lithium-ion battery (LIB) disassembly applications. Experimental results demonstrate its adaptability to dynamic collaborative tasks and generalization in generating strategies for unplanned operational variations within dynamic disassembly environments. This approach effectively overcomes various technical challenges to achieve autonomous collaboration in MHMRC systems through knowledge representation, task allocation, and collaborative optimization.
{"title":"GNN-LLM hybrid cognitive architectures for generative task adaptation in multi-human multi-robot collaborative disassembly","authors":"Xiaodong Tong, Ke Li, Jinsong Bao","doi":"10.1016/j.rcim.2025.103169","DOIUrl":"10.1016/j.rcim.2025.103169","url":null,"abstract":"<div><div>Traditional human-robot collaboration research has primarily focused on single human-robot dyads, yet faces significant challenges in addressing complex industrial scenarios characterized by concurrent multi-tasking, dynamic disturbances, and heterogeneous role coordination. Transitioning toward multi-human multi-robot collaboration (MHMRC) is crucial for achieving a significant leap in coordinated efficiency and manufacturing flexibility. To address this, we investigate a Hybrid Cognitive Digital Twin (HCDT) framework through generative knowledge-augmented paradigms. Our approach introduces a human-centric cognitive entity to generate task data and knowledge-driven strategies for MHMRC. This work demonstrates that integrating Large Language Models (LLMs) with Graph Neural Networks (GNNs) offers robust capabilities in comprehension, reasoning, ideally meeting MHMRC's requirements for handling unplanned operational variations as well as adapting to dynamic collaborative tasks. Furthermore, we demonstrate that compared to human-engineered precoding strategies, the HCDT-powered MHMRC system autonomously generates collaborative strategies for unscheduled tasks under more complex dynamic conditions and mission scenarios, enabling the execution of situations beyond conventional predefined patterns. The proposed methodology was validated in automotive lithium-ion battery (LIB) disassembly applications. Experimental results demonstrate its adaptability to dynamic collaborative tasks and generalization in generating strategies for unplanned operational variations within dynamic disassembly environments. This approach effectively overcomes various technical challenges to achieve autonomous collaboration in MHMRC systems through knowledge representation, task allocation, and collaborative optimization.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103169"},"PeriodicalIF":11.4,"publicationDate":"2025-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145314951","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-10-15DOI: 10.1016/j.rcim.2025.103166
Zi-Peng Chi , Qing-Hui Wang , Hai-Long Xie , Jian-Long Ni , A.Y.C. Nee , S.K. Ong
Currently planning the toolpath for robotic polishing of aero-engine impellers is still a challenging job due to its narrow and twisted processing channel and prone to various processing interferences. Inspired by the intelligent perception and adaptive decision-making ability of skilled workers, this work proposes a haptic-based human-robot collaborative (HRC) programming interface for robotic polishing of impellers to leverage the experience of skilled operators. With this interface, an HRC programming system is developed by integrating a haptic device, which enables operators to demonstrate a favorable trajectory in a realistic virtual reality (VR) environment by perceiving the polishing force and observing the polishing effect simulated by the system. To enhance the operator's hand-eye coordination ability during HRC programming, an intuitive workspace mapping algorithm between the haptic devices and robots is proposed. In addition, a flexible virtual fixture that can capture the operator's programming intention is proposed, which can adaptively impose appropriate motion and force constraints on the operator’s hand to facilitate interference avoidance and achieve the desired surface quality. The effectiveness and practicability of the proposed approach are validated by toolpath planning and robotic physical polishing experiments of impellers, which shows that the proposed method can reduce the operator’s cognitive load during HRC programming and enhance both programming efficiency and accuracy. Moreover, the method improves both the quality and consistency of polished surfaces since it combines both the advantages of human intelligence and expertise with the high movement accuracy of robots.
{"title":"Human-robot collaborative programming for robotic polishing of impeller using adaptive virtual fixtures and haptic interface","authors":"Zi-Peng Chi , Qing-Hui Wang , Hai-Long Xie , Jian-Long Ni , A.Y.C. Nee , S.K. Ong","doi":"10.1016/j.rcim.2025.103166","DOIUrl":"10.1016/j.rcim.2025.103166","url":null,"abstract":"<div><div>Currently planning the toolpath for robotic polishing of aero-engine impellers is still a challenging job due to its narrow and twisted processing channel and prone to various processing interferences. Inspired by the intelligent perception and adaptive decision-making ability of skilled workers, this work proposes a haptic-based human-robot collaborative (HRC) programming interface for robotic polishing of impellers to leverage the experience of skilled operators. With this interface, an HRC programming system is developed by integrating a haptic device, which enables operators to demonstrate a favorable trajectory in a realistic virtual reality (VR) environment by perceiving the polishing force and observing the polishing effect simulated by the system. To enhance the operator's hand-eye coordination ability during HRC programming, an intuitive workspace mapping algorithm between the haptic devices and robots is proposed. In addition, a flexible virtual fixture that can capture the operator's programming intention is proposed, which can adaptively impose appropriate motion and force constraints on the operator’s hand to facilitate interference avoidance and achieve the desired surface quality. The effectiveness and practicability of the proposed approach are validated by toolpath planning and robotic physical polishing experiments of impellers, which shows that the proposed method can reduce the operator’s cognitive load during HRC programming and enhance both programming efficiency and accuracy. Moreover, the method improves both the quality and consistency of polished surfaces since it combines both the advantages of human intelligence and expertise with the high movement accuracy of robots.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103166"},"PeriodicalIF":11.4,"publicationDate":"2025-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145314953","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The ultrasonic vibration-assisted milling is now widely used in the biomedical, aerospace and electronic manufacturing as its high adaptability and machining efficiency of difficult-to-cut materials. However, the primary challenges including incorrect numerical control (NC) programming code, complicated material removal procedure simulation, reliable in-process machining status monitoring, inevitable interference and collision of cutting tool with workpiece and worktable would limit the application of the ultrasonic vibration-assisted milling. To address these challenges, a digital twin-driven in-process monitoring system is proposed for the ultrasonic vibration-assisted milling process. The fundamental architecture design for the in-process monitoring system is established from the digital twin model for real-time motion control, real-time material removal procedure simulation, in-process physical state analysis, in-process fault diagnosis and historical machining reproduction during the ultrasonic vibration-assisted milling process. The geometric motion control is implemented by converting NC programming code and sensor data into real-time machining trajectories. Meanwhile, the real-time material removal process monitoring is developed with mesh visibility and spatial location relationship for dynamic geometric simulation. More importantly, the fault source is encoded for spindle speed, and the fault diagnosis of the ultrasonic vibration-assisted milling process can be realized by the optimized sparrow search algorithm with back propagation (SSA-BP) neural network. Moreover, the real-time physical state analysis is implemented by the radial basis function (RBF) interpolation and Socket protocols. The function of reproducing historical machining procedure is achieved by the timestamp of historical data in Unity3d script. The proposed monitoring system is validated on the established experiment platform. Correspondingly, the different modules consisting of geometric motion, physical state and data transmission are integrated, moreover, the system client covering client functionality, operational efficiency, and display performance is also tested, in which the efficient and stable operation of the developed in-process monitoring system of ultrasonic vibration-assisted milling can be ensured.
{"title":"A digital twin-driven in-process monitoring system for the ultrasonic vibration-assisted milling","authors":"Xuewei Zhang , Ting Shi , Xianzhen Huang , Tianbiao Yu","doi":"10.1016/j.rcim.2025.103168","DOIUrl":"10.1016/j.rcim.2025.103168","url":null,"abstract":"<div><div>The ultrasonic vibration-assisted milling is now widely used in the biomedical, aerospace and electronic manufacturing as its high adaptability and machining efficiency of difficult-to-cut materials. However, the primary challenges including incorrect numerical control (NC) programming code, complicated material removal procedure simulation, reliable in-process machining status monitoring, inevitable interference and collision of cutting tool with workpiece and worktable would limit the application of the ultrasonic vibration-assisted milling. To address these challenges, a digital twin-driven in-process monitoring system is proposed for the ultrasonic vibration-assisted milling process. The fundamental architecture design for the in-process monitoring system is established from the digital twin model for real-time motion control, real-time material removal procedure simulation, in-process physical state analysis, in-process fault diagnosis and historical machining reproduction during the ultrasonic vibration-assisted milling process. The geometric motion control is implemented by converting NC programming code and sensor data into real-time machining trajectories. Meanwhile, the real-time material removal process monitoring is developed with mesh visibility and spatial location relationship for dynamic geometric simulation. More importantly, the fault source is encoded for spindle speed, and the fault diagnosis of the ultrasonic vibration-assisted milling process can be realized by the optimized sparrow search algorithm with back propagation (SSA-BP) neural network. Moreover, the real-time physical state analysis is implemented by the radial basis function (RBF) interpolation and Socket protocols. The function of reproducing historical machining procedure is achieved by the timestamp of historical data in Unity3d script. The proposed monitoring system is validated on the established experiment platform. Correspondingly, the different modules consisting of geometric motion, physical state and data transmission are integrated, moreover, the system client covering client functionality, operational efficiency, and display performance is also tested, in which the efficient and stable operation of the developed in-process monitoring system of ultrasonic vibration-assisted milling can be ensured.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103168"},"PeriodicalIF":11.4,"publicationDate":"2025-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145314954","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-10-13DOI: 10.1016/j.rcim.2025.103164
Hongjiang Lu , Lilan Liu , Zenggui Gao , Yuyan Yao , Xinjie Cao , Jingwei Tang
Digital twins, by integrating physical entities with virtual models and combining real-time data with physical mechanisms for dynamic interaction and optimization, have become a crucial tool in structural health monitoring. However, existing digital twin models still face limitations in the depth of mechanism and data fusion, as well as in their spatiotemporal collaborative analysis capabilities, which results in poor predictive performance in complex dynamic environments and an inability to fully capture the global state evolution of structures. To address these challenges, this study proposes a spatiotemporal collaborative digital twin (SC-DT) approach for structural health monitoring, which integrates numerical simulation, machine learning, deep learning, surrogate model, and data processing and visualization techniques, enabling real-time monitoring and accurate prediction of structural health status. Using a six-axis robotic arm as an example, the principles and implementation process of the SC-DT method are detailed, and its effectiveness is validated through experiments. Additionally, by comparing the proposed physics-informed hybrid network (PIHN) model with other models, the superiority of the PIHN model in terms of accuracy and effectiveness is demonstrated. Compared to traditional finite element methods, the SC-DT method significantly reduces the time cost of structural performance analysis, achieving instantaneous predictions within 0.1 seconds in the six-axis robotic arm case study, thus providing a novel solution for real-time health monitoring of complex structures.
{"title":"Spatiotemporal collaborative digital twin structural health monitoring based on data mechanism fusion","authors":"Hongjiang Lu , Lilan Liu , Zenggui Gao , Yuyan Yao , Xinjie Cao , Jingwei Tang","doi":"10.1016/j.rcim.2025.103164","DOIUrl":"10.1016/j.rcim.2025.103164","url":null,"abstract":"<div><div>Digital twins, by integrating physical entities with virtual models and combining real-time data with physical mechanisms for dynamic interaction and optimization, have become a crucial tool in structural health monitoring. However, existing digital twin models still face limitations in the depth of mechanism and data fusion, as well as in their spatiotemporal collaborative analysis capabilities, which results in poor predictive performance in complex dynamic environments and an inability to fully capture the global state evolution of structures. To address these challenges, this study proposes a spatiotemporal collaborative digital twin (SC-DT) approach for structural health monitoring, which integrates numerical simulation, machine learning, deep learning, surrogate model, and data processing and visualization techniques, enabling real-time monitoring and accurate prediction of structural health status. Using a six-axis robotic arm as an example, the principles and implementation process of the SC-DT method are detailed, and its effectiveness is validated through experiments. Additionally, by comparing the proposed physics-informed hybrid network (PIHN) model with other models, the superiority of the PIHN model in terms of accuracy and effectiveness is demonstrated. Compared to traditional finite element methods, the SC-DT method significantly reduces the time cost of structural performance analysis, achieving instantaneous predictions within 0.1 seconds in the six-axis robotic arm case study, thus providing a novel solution for real-time health monitoring of complex structures.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103164"},"PeriodicalIF":11.4,"publicationDate":"2025-10-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145314956","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Disassembly tasks in human–robot collaboration (HRC) environments present safety challenges due to hazardous materials, control system variability, and physically demanding operator tasks. To address these challenges, we propose an AI-augmented risk assessment framework integrating System-Theoretic Process Analysis (STPA) and Failure Mode and Effects Analysis (FMEA). This framework is implemented in four configurations: Term Frequency–Inverse Document Frequency (TF-IDF), Fine-tuned Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and RAG with a structured Knowledge Graph (KG) built from safety standards. The system supports real-time, standards-compliant safety reasoning by generating interpretable, context-specific recommendations. We evaluate these configurations across GPT-3.5 TURBO, GPT-4o, GPT-4.1, and open-source LLMs Qwen2.5 (3B) and Ministral (3B). Among all, RAG+KG with GPT-4.1 achieved the highest results across language-based metrics (BLEU: 68.3, ROUGE-L: 72.0, Semantic Similarity: 81.1, BERTScore (F1): 90.0) and safety-specific metrics (Hazard Recall: 92, Compliance Precision: 97, Safety Violation Rate: zero). Six safety-oriented metrics were introduced to assess compliance, hazard coverage, interpretability, and robustness. A case study on electrical vehicle (EV) battery module disassembly demonstrated the system’s effectiveness in identifying unsafe control actions, tracing failure modes, and recommending targeted mitigation strategies for mechanical, electrical, and chemical hazards, and ergonomic considerations. This framework offers a scalable, explainable approach to real-time safety analysis, advancing AI-enabled risk assessment in dynamic HRC disassembly tasks and supporting the vision of human-centered Industry 5.0 manufacturing.
{"title":"Proactive safety reasoning in human-robot collaboration in disassembly through LLM-augmented STPA and FMEA","authors":"Morteza Jalali Alenjareghi , Fardin Ghorbani , Samira Keivanpour , Yuvin Adnarain Chinniah , Sabrina Jocelyn","doi":"10.1016/j.rcim.2025.103162","DOIUrl":"10.1016/j.rcim.2025.103162","url":null,"abstract":"<div><div>Disassembly tasks in human–robot collaboration (HRC) environments present safety challenges due to hazardous materials, control system variability, and physically demanding operator tasks. To address these challenges, we propose an AI-augmented risk assessment framework integrating System-Theoretic Process Analysis (STPA) and Failure Mode and Effects Analysis (FMEA). This framework is implemented in four configurations: Term Frequency–Inverse Document Frequency (TF-IDF), Fine-tuned Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and RAG with a structured Knowledge Graph (KG) built from safety standards. The system supports real-time, standards-compliant safety reasoning by generating interpretable, context-specific recommendations. We evaluate these configurations across GPT-3.5 TURBO, GPT-4o, GPT-4.1, and open-source LLMs Qwen2.5 (3B) and Ministral (3B). Among all, RAG+KG with GPT-4.1 achieved the highest results across language-based metrics (BLEU: 68.3, ROUGE-L: 72.0, Semantic Similarity: 81.1, BERTScore (F1): 90.0) and safety-specific metrics (Hazard Recall: 92, Compliance Precision: 97, Safety Violation Rate: zero). Six safety-oriented metrics were introduced to assess compliance, hazard coverage, interpretability, and robustness. A case study on electrical vehicle (EV) battery module disassembly demonstrated the system’s effectiveness in identifying unsafe control actions, tracing failure modes, and recommending targeted mitigation strategies for mechanical, electrical, and chemical hazards, and ergonomic considerations. This framework offers a scalable, explainable approach to real-time safety analysis, advancing AI-enabled risk assessment in dynamic HRC disassembly tasks and supporting the vision of human-centered Industry 5.0 manufacturing.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103162"},"PeriodicalIF":11.4,"publicationDate":"2025-10-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145314955","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-10-11DOI: 10.1016/j.rcim.2025.103167
Jianpeng Chen , Sihan Huang , Xiaowen Wang , Pengfei Wang , Jiahao Zhu , Zhe Xu , Guoxin Wang , Yan Yan , Lihui Wang
With the advent of Industry 5.0, human-centric smart manufacturing is becoming a new paradigm for industrial transformation. Human-robot collaboration (HRC) is the hot topic of human-centric smart manufacturing. The emergence of large language model (LLM) provides significant opportunity for collaborative robot to promote the autonomous collaboration ability, which brings HRC into new era driven by embodied intelligence and more powerful robot. Therefore, a dynamic autonomous collaboration method inspired from looking-thinking-doing chain of human operators is proposed for human-like collaborative robot (HLCobot) in human-centric smart manufacturing based on multimodal large language model (MLLM), where perception-decision-execution coordination mechanism is constructed to appropriately distribute the abilities of MLLM in the dynamic operation chain of HRC. Firstly, a brain-inspired architecture with the integration of perception hub, decision hub, and execution hub is designed for dynamic autonomous collaboration. Secondly, the abilities of perception, decision, execution of HLCobot are realized by integrating MLLM, where the HLCobot can actively recognize the dynamic changes of HRC scenario by mimicking human operator and execute the correct motions to complete the necessary collaborative task autonomously. Additionally, a coordination mechanism among the agents of perception, decision, and execution is put forward to proceed the collaborative task smoothly. Finally, a case study of engine assembly is provided to demonstrate the effectiveness of the proposed method.
随着工业5.0的到来，以人为中心的智能制造正在成为产业转型的新范式。人机协作（Human-robot collaboration， HRC）是以人为中心的智能制造的热点。大语言模型（large language model， LLM）的出现为协作机器人自主协作能力的提升提供了重要契机，使HRC进入了由具身智能和更强大的机器人驱动的新时代。因此，基于多模态大语言模型（multimodal large language model， MLLM），针对以人为中心的智能制造中的类人协作机器人（HLCobot），提出了一种受人类操作者“看-想-做”链启发的动态自主协作方法，构建感知-决策-执行协调机制，将MLLM的能力在HRC的动态操作链中合理分配。首先，设计了一种集成感知中心、决策中心和执行中心的基于大脑的动态自主协作架构；其次，通过整合MLLM实现HLCobot的感知、决策、执行能力，HLCobot可以通过模仿人类操作员，主动识别HRC场景的动态变化，并执行正确的动作，自主完成必要的协同任务。在此基础上，提出了感知、决策和执行agent之间的协调机制，以保证协同任务的顺利进行。最后，以发动机总成为例，验证了该方法的有效性。
{"title":"Perception-decision-execution coordination mechanism driven dynamic autonomous collaboration method for human-like collaborative robot based on multimodal large language model","authors":"Jianpeng Chen , Sihan Huang , Xiaowen Wang , Pengfei Wang , Jiahao Zhu , Zhe Xu , Guoxin Wang , Yan Yan , Lihui Wang","doi":"10.1016/j.rcim.2025.103167","DOIUrl":"10.1016/j.rcim.2025.103167","url":null,"abstract":"<div><div>With the advent of Industry 5.0, human-centric smart manufacturing is becoming a new paradigm for industrial transformation. Human-robot collaboration (HRC) is the hot topic of human-centric smart manufacturing. The emergence of large language model (LLM) provides significant opportunity for collaborative robot to promote the autonomous collaboration ability, which brings HRC into new era driven by embodied intelligence and more powerful robot. Therefore, a dynamic autonomous collaboration method inspired from looking-thinking-doing chain of human operators is proposed for human-like collaborative robot (HLCobot) in human-centric smart manufacturing based on multimodal large language model (MLLM), where perception-decision-execution coordination mechanism is constructed to appropriately distribute the abilities of MLLM in the dynamic operation chain of HRC. Firstly, a brain-inspired architecture with the integration of perception hub, decision hub, and execution hub is designed for dynamic autonomous collaboration. Secondly, the abilities of perception, decision, execution of HLCobot are realized by integrating MLLM, where the HLCobot can actively recognize the dynamic changes of HRC scenario by mimicking human operator and execute the correct motions to complete the necessary collaborative task autonomously. Additionally, a coordination mechanism among the agents of perception, decision, and execution is put forward to proceed the collaborative task smoothly. Finally, a case study of engine assembly is provided to demonstrate the effectiveness of the proposed method.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103167"},"PeriodicalIF":11.4,"publicationDate":"2025-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145262007","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-10-10DOI: 10.1016/j.rcim.2025.103159
Jintao Xue, Xiao Li, Nianmin Zhang
In advanced manufacturing systems, humans and robots collaborate to conduct the production process. Effective task planning and allocation (TPA) is crucial for achieving high production efficiency, yet it remains challenging in complex and dynamic manufacturing environments. The dynamic nature of humans and robots, particularly the need to consider spatial information (e.g., humans’ real-time position and the distance they need to move to complete a task), substantially complicates TPA. To address the above challenges, we decompose production tasks into manageable subtasks. We then implement a real-time hierarchical human–robot TPA algorithm, including a high-level agent for task planning and a low-level agent for task allocation. For the high-level agent, we propose an efficient buffer-based deep Q-learning method (EBQ), which reduces training time and enhances performance in production problems with long-term and sparse reward challenges. For the low-level agent, a path planning-based spatially aware method (SAP) is designed to allocate tasks to the appropriate human–robot resources, thereby achieving the corresponding sequential subtasks. We conducted experiments on a complex real-time production process in a 3D simulator. The results demonstrate that our proposed EBQ&SAP method effectively addresses human–robot TPA problems in complex and dynamic production processes.
{"title":"A hierarchical spatial–aware algorithm with efficient reinforcement learning for human–robot task planning and allocation in production","authors":"Jintao Xue, Xiao Li, Nianmin Zhang","doi":"10.1016/j.rcim.2025.103159","DOIUrl":"10.1016/j.rcim.2025.103159","url":null,"abstract":"<div><div>In advanced manufacturing systems, humans and robots collaborate to conduct the production process. Effective task planning and allocation (TPA) is crucial for achieving high production efficiency, yet it remains challenging in complex and dynamic manufacturing environments. The dynamic nature of humans and robots, particularly the need to consider spatial information (e.g., humans’ real-time position and the distance they need to move to complete a task), substantially complicates TPA. To address the above challenges, we decompose production tasks into manageable subtasks. We then implement a real-time hierarchical human–robot TPA algorithm, including a high-level agent for task planning and a low-level agent for task allocation. For the high-level agent, we propose an efficient buffer-based deep Q-learning method (EBQ), which reduces training time and enhances performance in production problems with long-term and sparse reward challenges. For the low-level agent, a path planning-based spatially aware method (SAP) is designed to allocate tasks to the appropriate human–robot resources, thereby achieving the corresponding sequential subtasks. We conducted experiments on a complex real-time production process in a 3D simulator. The results demonstrate that our proposed EBQ&SAP method effectively addresses human–robot TPA problems in complex and dynamic production processes.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103159"},"PeriodicalIF":11.4,"publicationDate":"2025-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145261988","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-10-09DOI: 10.1016/j.rcim.2025.103160
Atefeh Rajabi-Kafshgar , Mostafa Hajiaghaei-Keshteli , Mohammad Reza Mohammad Aliha
Recent developments in cloud manufacturing (CMg) have highlighted the need for efficient task scheduling and resource allocation in distributed and dynamic environments. To the best of our knowledge, existing studies have not considered dynamic events such as task cancellation, which can lead to resource inefficiencies and disrupt the initial schedule. To address this gap, this paper introduces a novel dynamic task scheduling and service allocation (DTSSA) problem in CMg that considers task cancellation. The proposed model considers logistics time and different arrival times, which directly impact the tasks’ completion times. Furthermore, a reinforcement learning-based genetic algorithm is developed to tackle the NP-hardness of the model and solve medium- and large-scale problems in a reasonable time. The algorithm dynamically selects search operators using the Q-learning algorithm and applies a ε-greedy approach to improve search capabilities. In this regard, first, the metaheuristic algorithms’ parameters are tuned by the Taguchi method. The proposed algorithms were evaluated using 30 benchmark instances from the literature, as well as example cases inspired by existing studies. Next, the mathematical model is evaluated by implementing small-scale examples using GAMS software. Then, the algorithms are compared with not only some well-known metaheuristic algorithms but also recently developed metaheuristic algorithms using statistical tests and several test problems of different sizes. Additionally, results show that the rescheduling problem provides up to 8.7% better solutions on average than the initial schedule. Lastly, the model's sensitivity analysis reveals that the longer the processing time and logistic time, the longer the maximum completion time for scheduling and rescheduling.
{"title":"A reinforcement learning-based metaheuristic approach to address the dynamic scheduling problem in cloud manufacturing with task cancellation","authors":"Atefeh Rajabi-Kafshgar , Mostafa Hajiaghaei-Keshteli , Mohammad Reza Mohammad Aliha","doi":"10.1016/j.rcim.2025.103160","DOIUrl":"10.1016/j.rcim.2025.103160","url":null,"abstract":"<div><div>Recent developments in cloud manufacturing (CMg) have highlighted the need for efficient task scheduling and resource allocation in distributed and dynamic environments. To the best of our knowledge, existing studies have not considered dynamic events such as task cancellation, which can lead to resource inefficiencies and disrupt the initial schedule. To address this gap, this paper introduces a novel dynamic task scheduling and service allocation (DTSSA) problem in CMg that considers task cancellation. The proposed model considers logistics time and different arrival times, which directly impact the tasks’ completion times. Furthermore, a reinforcement learning-based genetic algorithm is developed to tackle the NP-hardness of the model and solve medium- and large-scale problems in a reasonable time. The algorithm dynamically selects search operators using the Q-learning algorithm and applies a <em>ε</em>-greedy approach to improve search capabilities. In this regard, first, the metaheuristic algorithms’ parameters are tuned by the Taguchi method. The proposed algorithms were evaluated using 30 benchmark instances from the literature, as well as example cases inspired by existing studies. Next, the mathematical model is evaluated by implementing small-scale examples using GAMS software. Then, the algorithms are compared with not only some well-known metaheuristic algorithms but also recently developed metaheuristic algorithms using statistical tests and several test problems of different sizes. Additionally, results show that the rescheduling problem provides up to 8.7% better solutions on average than the initial schedule. Lastly, the model's sensitivity analysis reveals that the longer the processing time and logistic time, the longer the maximum completion time for scheduling and rescheduling.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103160"},"PeriodicalIF":11.4,"publicationDate":"2025-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145261992","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-10-09DOI: 10.1016/j.rcim.2025.103158
Xu Tang, Jixiang Yang, Han Ding
Environmental adaptability is a key challenge in robotic operation, while the limited compliance of parallel manipulators hinders their application in variable-stiffness environments. This paper proposes a novel three-degree-of-freedom parallel electromagnetic variable stiffness manipulator (PEVSM) that actively adapts to the environment through self-stiffness modulation. PEVSM integrates an unconstrained variable stiffness limb driven by an electromagnetic spring and three compliant limbs actuated by Lorentz motors, enabling active and continuous stiffness modulation. Based on the established kinematic and stiffness models, a hybrid force-position-stiffness control framework is developed, integrating enhanced fractional-order adaptive impedance control and a stiffness controller based on deep deterministic policy gradient with multi-source feedback to achieve precise and compliant force regulation. For applying robotic grinding to low-stiffness workpieces, a force–deformation model and a force compensation strategy are introduced to mitigate deformation effects and improve material removal accuracy. The robotic grinding platform with PEVSM is constructed, demonstrating its advantages on improve force control and material removal accuracy in compliant grinding.
{"title":"Design and control of a parallel electromagnetic variable stiffness manipulator for robotic compliant grinding","authors":"Xu Tang, Jixiang Yang, Han Ding","doi":"10.1016/j.rcim.2025.103158","DOIUrl":"10.1016/j.rcim.2025.103158","url":null,"abstract":"<div><div>Environmental adaptability is a key challenge in robotic operation, while the limited compliance of parallel manipulators hinders their application in variable-stiffness environments. This paper proposes a novel three-degree-of-freedom parallel electromagnetic variable stiffness manipulator (PEVSM) that actively adapts to the environment through self-stiffness modulation. PEVSM integrates an unconstrained variable stiffness limb driven by an electromagnetic spring and three compliant limbs actuated by Lorentz motors, enabling active and continuous stiffness modulation. Based on the established kinematic and stiffness models, a hybrid force-position-stiffness control framework is developed, integrating enhanced fractional-order adaptive impedance control and a stiffness controller based on deep deterministic policy gradient with multi-source feedback to achieve precise and compliant force regulation. For applying robotic grinding to low-stiffness workpieces, a force–deformation model and a force compensation strategy are introduced to mitigate deformation effects and improve material removal accuracy. The robotic grinding platform with PEVSM is constructed, demonstrating its advantages on improve force control and material removal accuracy in compliant grinding.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"98 ","pages":"Article 103158"},"PeriodicalIF":11.4,"publicationDate":"2025-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145261990","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号