Towards safe motion planning for industrial human-robot interaction: A co-evolution approach based on human digital twin and mixed reality

Abstract

Advanced human-robot interaction (HRI) is essential for the next-generation human-centric manufacturing mode such as “Industry 5.0”. Despite recent mutual cognitive approaches can enhance the understanding and collaboration between humans and robots, these methods often rely on predefined rules and are limited in adapting to new tasks or changes of the working environment. These limitations can hinder the popularization of collaborative robots in dynamic manufacturing environments, where tasks can be highly variable, and unforeseen operational changes frequently occur. To address these challenges, we propose a co-evolution approach for the safe motion planning of industrial human-robot interaction. The core idea is to promote the evolution of human worker’s safe operation cognition as well as the evolution of robot’s safe motion planning strategy in a unified and continuous framework by leveraging human digital twin (HDT) and mixed reality (MR) technologies. Specifically, HDT captures real-time human behaviors and postures, which enables robots to adapt dynamically to the changes of human behavior and environment. HDT also refines deep reinforcement learning (DRL)-based motion planning, allowing robots to continuously learn from human actions and update their motion strategies. On the other hand, MR superimposes rich information regarding the tasks and robot in the physical world, helping human workers better understand and adapt to robot’s actions. MR also provides intuitive gesture-based user interface, further improving the smoothness of human-robot interaction. We validate the proposed approach’s effectiveness with evaluations in realistic manufacturing scenarios, demonstrating its potential to advance HRI practice in the context of smart manufacturing.