Robotics and Computer-integrated Manufacturing最新文献

Existing augmented reality (AR) assembly mainly provides visual instructions for operators from a first-person perspective, and it is hard to share individual working intents for co-located workers on the shop floor, especially for large-scale product assembly task that requires multiple operators working together. To bridge this gap for practical deployments, this paper proposes Co²iAR, a co-located audio-visual enabled mobile collaborative AR assembly. Firstly, according to the stereo visual-inertial fusion strategy, robust and accurate self-contained motion tracking is achieved for the resource-constrained mobile AR platform, followed by a co-located alignment from multiple mobile AR clients on the shop floor. Then, a lightweight text-aware network for online wiring harness character recognition is proposed, as well as the audio-based confirming strategy, enabling natural audio-visual interaction among co-located workers within a shared immersive workplace, which can also monitor the current wiring assembly status and activate the step-by-step tutorials automatically. The novelty of this work is focused on the deployment of audio-visual aware interaction using the same device that is being used to deploy the co-located collaborative AR work instructions, establishing shared operating intents among multiple co-located workers. Finally, comprehensive experiments are carried out on the collaborative performance among multiple AR clients, and results illustrate that the proposed Co²iAR can alleviate the cognitive load and achieve superior performance for the co-located AR assembly tasks, providing a more human-centric collaborative assembly performance.

Assembly processes for complex products primarily involve manual assembly and often encounter various disruptive events, such as the insertion of new orders, order cancellations, task adjustments, workers absences, and job rotations. The dynamic scheduling problem for complex product assembly workshops requires consideration of trigger events and time nodes for rescheduling, as well as the allocations of multi-skilled and multi-level workers. The application of digital twin technology in smart manufacturing enables managers to more effectively monitor and control disruptive events and production factors on the production site. Therefore, a dynamic scheduling strategy based on digital twin technology is proposed to enable real-time monitoring of dynamic events in the assembly workshop, triggering rescheduling when necessary, adjusting task processing sequences and team composition accordingly, and establishing a corresponding dynamic scheduling integer programming model. Additionally, based on NSGA-II, an improved multi-objective evolutionary algorithm (IMOEA) is proposed, which utilizes the maximum completion time as the production efficiency indicator and the time deviation before and after rescheduling as the production stability indicator. Three new population initialization rules are designed, and the optimal parameter combination for these rules is determined. Finally, the effectiveness of the scheduling strategy is verified through the construction of a workshop digital twin system.

Region-based robotic machining is considered an effective strategy for automatically repairing paint film defects compared to conventional global machining. However, this process faces challenges due to irregularities in defect position, shape, and size. To overcome these challenges, this paper proposes a model-enabled robotic machining framework for repairing paint film defects by leveraging the workpiece model as an enabling means. Within the system framework, an improved YOLOv5 algorithm is presented at first to enhance the visual detection accuracy of paint film defects in terms of network structure and loss function. Additionally, a target positioning method based on the pixel-point inverse projection technology is developed to map the 2D defect detection results onto the workpiece 3D model, which primarily aims at obtaining the orientation information through the connection between the monocular vision unit and the model. Finally, an optimal tool deployment strategy by virtue of the least projection coverage circle is proposed to determine the least machined position as well as the shortest robot path by constructing the mapping between the defects and the tool operation size. The constructed system framework is verified effective and practical by the experiments of region-based robotic grinding and repairing of paint film defects on high-speed train (HST) body sidewalls.

With the rapid advancements in three-dimensional (3D) printing, researchers have shifted their focus towards the mechanical systems and methods used in this field. While Fused Deposition Modelling (FDM) remains the dominant method, alternative printing methods such as Spatial 3DP (S-3DP) have emerged. However, the majority of existing research on 3D printing technology has been emphasizing offline control, which lacks the capability to dynamically adjust the printing path in real time. Such an limitation has resulted in a decrease in printing efficiency. Therefore, this paper proposes a human-robot interaction (HRI) method based on real-time gesture control for Robotic Spatial 3DP (RS-3DP). This method incorporates utilization of YOLOv5 and Mediapipe algorithms to recognize gestures and convert the gesture information into real-time robot operations. Results show that this approach offers a feasible solution to address the issue of discontinuous S-3DP nodes because it achieves a gesture-controlled robot movement accuracy of 91 % and an average system response time of approximately 0.54 s. The proposed HRI method represents a pioneering advancement in real-time control for RS-3DP, thereby paving the way for further exploration and development in this field.

Industry 5.0 paradigm has renewed interest in the human sphere, emphasizing the importance of workers’ well-being in manufacturing activities. In such context, collaborative robotics originated as a technology to support humans in tiring and repetitive tasks. This study investigates the effects of assembly complexity in Human-Robot collaboration using physiological indicators of cognitive effort. In a series of experiments, participants performed assembly processes of different products with varying complexity, in two modalities: manually and with cobot assistance. Physiological measures, including skin conductance, heart rate variability and eye-tracking metrics were collected. The analysis of physiological signals showed trends suggesting the impact of assembly complexity and cobot support. One key finding of the study is that a single physiological signal usually may not provide a complete understanding of cognitive load. Therefore, a holistic approach should be followed. This approach highlighted the importance of considering multiple measures simultaneously to accurately assess workers’ well-being in industrial environments.

With the recent vision of Industry 5.0, the cognitive capability of robots plays a crucial role in advancing proactive human–robot collaborative assembly. As a basis of the mutual empathy, the understanding of a human operator’s intention has been primarily studied through the technique of human action recognition. Existing deep learning-based methods demonstrate remarkable efficacy in handling information-rich data such as physiological measurements and videos, where the latter category represents a more natural perception input. However, deploying these methods in new unseen assembly scenarios requires first collecting abundant case-specific data. This leads to significant manual effort and poor flexibility. To deal with the issue, this paper proposes a novel cross-domain few-shot learning method for data-efficient multimodal human action recognition. A hierarchical data fusion mechanism is designed to jointly leverage the skeletons, RGB images and depth maps with complementary information. Then a temporal CrossTransformer is developed to enable the action recognition with very limited amount of data. Lightweight domain adapters are integrated to further improve the generalization with fast finetuning. Extensive experiments on a real car engine assembly case show the superior performance of proposed method over state-of-the-art regarding both accuracy and finetuning efficiency. Real-time demonstrations and ablation study further indicate the potential of early recognition, which is beneficial for the robot procedures generation in practical applications. In summary, this paper contributes to the rarely explored realm of data-efficient human action recognition for proactive human–robot collaboration.

Robotic additive manufacturing using a cold spray deposition head attached to a robotic arm can deposit material in a solid state with deposition rates in kilogrammes per hour. Under such a high deposition rate, the complicated interplay between the robot’s motion, gun standoff distance, spray angle, overlapping, and the interaction of supersonic powder particles with a growing structure could cause overabundance or deficiency of material build-up. Over time, the accumulation of these discrepancies can negatively affect the overall shape and size of the final manufactured object. In-process spatio-temporal 3D reconstruction, also known as 4D reconstruction, could allow for early detection of deviations from the design, thus providing the opportunity to rectify at an early stage, making the process more robust, efficient and productive. However, in-process model reconstruction is challenging due to the dynamic nature of the scene (e.g. sensor and object relative movements), the three-dimensional growth of a time-varying build object, the textureless nature of build surfaces, and its computational complexity. We propose a real-time, in-process 4D reconstruction framework for free-form additive manufacturing processes, such as cold spray that deals with a real-time dynamic and evolving scene built by incremental deposition of materials. In our approach, temporal point clouds from three cameras are acquired and segmented to extract the region of interest (build object). The subsequent multi-temporal and multi-camera registration of the segmented 3D data is addressed by combining geometrically constrained Fiducial marker tracking and plane-based registration without drift accumulation. Finally, the registered point clouds are fused via voxel fusion of growing parts to reconstruct the 3D model of the object with smoothened surfaces. The proposed solution is deployed and verified in a robotic cold spray cell with different test scenarios and shape complexities.

As advancements in transportation equipment intelligence continue, the job shop scheduling problem integrating finite transportation resources (JSPIFTR) has attracted considerable attention. Within the domain of shop scheduling, the neighborhood structure serves as a cornerstone for enabling intelligent optimization algorithms to effectively navigate and discover optimal solutions. However, current algorithms for JSPIFTR rely on generalized neighborhood structures, which incorporate operators like insertion and swap. While these structures are tailored to the encoding vectors, their utilization often leads to suboptimal optimization efficacy. To address this limitation, this paper introduces novel neighborhood structures specifically designed to the distinctive properties of JSPIFTR. These innovative structures leverage the intrinsic structural information in integrated scheduling, thereby enhancing the optimization effectiveness of the algorithm. Firstly, two theorems are presented to demonstrate the feasibility of the neighborhood solution. Secondly, different neighborhood structures for critical transportation and processing tasks are subsequently designed based on the analysis of the problem properties and constraints. Thirdly, an efficient fast evaluation method is developed to expediently calculate the objective value of the neighborhood solution. Finally, the novel neighborhood structures are combined with the tabu search (TS_NNS) and compared with other state-of-the-art methods on EX and NEX benchmarks. The comparative results demonstrate the remarkable performance of the neighborhood structure, with the TS_NNS enhancing the best solutions across 23 instances.

Industrial robots are promising and competitive alternatives for performing machining operations due to their advantages of good mobility, high flexibility and low cost. However, the application of industrial robots in the field of high-precision machining such as grinding is hugely limited by the characteristic of weak stiffness. Aiming at this problem, a whole-path posture optimization method of robotic grinding based on multi-performance evaluation indices is proposed in this paper. Firstly, a kinematic performance evaluation index is utilized to directly refine the regions of the robot workspace. Secondly, a stiffness performance evaluation index comprehensively considering the characteristics of grinding process is put forward. Simultaneously, a space conversion method is proposed to convert the stiffness index from the robot end to the tool end, and then a task-oriented flexibility ellipsoid on the tool-workpiece contact point is established. Furtherly, on these bases, aiming for the motion smoothness and the overall maximum stiffness of the robot in the whole grinding path, and taking the performance of the robot body as the constraint synergistically, an optimization model is established to optimize the posture of the robot. Finally, three groups of comparative grinding experiments are carried out on a KUKA kr210–2 robotic grinding platform. The results demonstrate that by using the posture optimization algorithm proposed in this paper, a better comprehensive performance including stiffness and motion smoothness in the whole grinding path can be achieved, and the workpiece after grinding has a higher removal depth and a better consistency, whose roughness has also been enhanced. These phenomenons indicate that the proposed method can significantly improve the accuracy and stability of grinding, thereby the effectiveness of this method is verified.