Robotics and Computer-integrated Manufacturing最新文献

Map construction is the initial step of mobile robots for their localization, navigation, and path planning in unknown environments. Considering the human-robot collaboration (HRC) scenarios in modern manufacturing, where the human workers’ capabilities are closely integrated with the efficiency and precision of robots in the same workspace, a map integrating geometric and semantic information is considered as the technical foundation for intelligent interactions between human workers and robots, such as motion planning, reasoning, and context-aware decision-making. Although different map construction methods have been proposed for mobile robots’ perception in the working environment, it is still a challenging task when applied to such human-robot collaborative manufacturing scenarios to achieve the afore-mentioned intelligent interactions between human workers and robots due to the poor integration of semantic information in the constructed map. On the one hand, due to the lack of ability for differentiating the dynamic objects, the mobile robot might sometimes wrongly use the dynamic objects as the spatial references to calculate the pose transformation between the two successive frames, which negatively affects the accuracy of the robot's localization and pose estimation. On the other hand, the map that integrates both the geometric and semantic information can hardly be constructed in real-time, which cannot provide an effective support for the real-time reasoning and decision making during the human-robot collaboration process.

This study proposes a novel map construction approach containing semantic information generation, geometric information generation, and semantic & geometric information fusion modules, which enables the integration of the semantic and geometric information in the constructed map. First, the semantic information generation module analyzes the captured images of the dynamic working environment, eliminates the features of dynamic objects, and generates the semantic information of the static objects. Meanwhile, the geometric information generation module is adopted to generate the accurate geometric information of the robot's motion plane by using the environment data. Finally, a map integrating semantic and geometric information in real-time can be constructed by the semantic & geometric fusion module. The experimental results demonstrate the effectiveness of the proposed semantic map construction approach.

In practice, the process of robot assembly in an unstructured environment faces difficulties due to the presence of unpredictable environmental errors related to vision and pose. Therefore, to minimize the uncertain environmental errors during the robotic assembly process in an unstructured environment, several studies have considered a reinforcement learning (RL)-based approach. However, if assembly parts are changed, it becomes difficult to apply RL-based methods to assemble various parts because additional learning may be required. Especially in the case of connector assembly, fine-tuning is essential because the shape changes depending on the type of connector. In this study, we propose a similar assembly state discriminator that transforms the state information (force, velocity, and RGB image) of reinforcement learning into generalized features to respond various types of connector assembly tasks. This method processes the data to include essential features for assembly regardless of connector type. By learning the RL model with the processed data using this method, the RL model trained for a specific connector can be efficiently applied to other types of connectors without fine-tuning. The assembly success rate for the 7 types of connectors (Harting, HDMI, USB, power, air jack, banana plug and PCIE) using the proposed method was demonstrated to be over 96 %.

Additive Manufacturing (AM) has revolutionized the production landscape by enabling on-demand customized manufacturing. However, the efficient management of dynamic AM orders poses significant challenges for production planning and scheduling. This paper addresses the dynamic scheduling problem considering batch processing, random order arrival and machine eligibility constraints, aiming to minimize total tardiness in a parallel non-identical AM machine environment. To tackle this problem, we propose the out-of-order enabled dueling deep Q network (O3-DDQN) approach. In the proposed approach, the problem is formulated as a Markov decision process (MDP). Three-dimensional features, encompassing dynamic orders, AM machines, and delays, are extracted using a ‘look around’ method to represent the production status at a rescheduling point. Additionally, five novel composite scheduling rules based on the out-of-order principle are introduced for selection when an AM machine completes processing or a new order arrives. Moreover, we design a reward function that is strongly correlated with the objective to evaluate the agent’s chosen action. Experimental results demonstrate the superiority of the O3-DDQN approach over single scheduling rules, randomly selected rules, and the classic DQN method. The average improvement rate of performance reaches 13.09% compared to composite scheduling rules and random rules. Additionally, the O3-DDQN outperforms the classic DQN agent with a 6.54% improvement rate. The O3-DDQN algorithm improves scheduling in dynamic AM environments, enhancing productivity and on-time delivery. This research contributes to advancing AM production and offers insights into efficient resource allocation.

Manufacturing is transformed towards smart manufacturing, entering a new data-driven era fueled by digital technologies. The resulting Smart Manufacturing Systems (SMS) gather extensive amounts of diverse data, thanks to the growing number of sensors and rapid advances in sensing technologies. Among the various data types available in SMS settings, time-series data plays a pivotal role. Hence, Time-Series Classification (TSC) emerges as a crucial task in this domain. Over the past decade, researchers have introduced numerous methods for TSC, necessitating not only algorithmic development and analysis but also validation and empirical comparison. This dual approach holds substantial value for practitioners by streamlining choices and revealing insights into models’ strengths and weaknesses. The objective of this study is to fill this gap by providing a rigorous experimental evaluation of the state-of-the-art Machine Learning (ML) and Deep Learning (DL) algorithms for TSC tasks in manufacturing and industrial settings. We first explored and compiled a comprehensive list of more than 92 state-of-the-art algorithms from both TSC and manufacturing literature. Following this, we methodologically selected the 36 most representative algorithms from this list. To evaluate their performance across various manufacturing classification tasks, we curated a set of 22 manufacturing datasets, representative of different characteristics that cover diverse manufacturing problems. Subsequently, we implemented and evaluated the algorithms on the manufacturing benchmark datasets, and analyzed the results for each dataset. Based on the results, ResNet, DrCIF, InceptionTime, and ARSENAL emerged as the top-performing algorithms, boasting an average accuracy of over 96.6 % across all 22 manufacturing TSC datasets. These findings underscore the robustness, efficiency, scalability, and effectiveness of convolutional kernels in capturing temporal features in time-series data collected from manufacturing systems for TSC tasks, as three out of the top four performing algorithms leverage these kernels for feature extraction. Additionally, LSTM, BiLSTM, and TS-LSTM algorithms deserve recognition for their effectiveness in capturing features within manufacturing time-series data using RNN-based structures.

Vision-based weld seam tracking has become one of the key technologies to realize intelligent robotic welding, and weld deviation detection is an essential step. However, accurate and robust detection of weld deviations during the microwelding of ultrathin metal foils remains a significant challenge. This challenge can be attributed to the fusion zone at the mesoscopic scale and the complex time-varying interference (pulsed arcs and reflected light from the workpiece surface). In this paper, an intelligent seam tracking approach for foils joining based on spatial–temporal deep learning from molten pool serial images is proposed. More specifically, a microscopic passive vision sensor is designed to capture molten pool and seam trajectory images under pulsed arc lights. A 3D convolutional neural network (3DCNN) and long short-term memory (LSTM)-based welding torch offset prediction network (WTOP-net) is established to implement highly accurate deviation prediction by capturing long-term dependence of spatial–temporal features. Then, expert knowledge is further incorporated into the spatio-temporal features to improve the robustness of the model. In addition, the slime mould algorithm (SMA) is used to prevent local optima and improve accuracy, efficiency of WTOP-net. The experimental results indicate that the maximum error detected by our method fluctuates within $\pm$ 0.08 mm and the average error is within $\pm$ 0.011 mm when joining two 0.12 mm thickness stainless steel diaphragms. The proposed approach provides a basis for automated robotic seam tracking and intelligent precision manufacturing of ultrathin sheets welded components in aerospace and other fields.

This paper addresses the crucial aspect of position error modeling and compensation in industrial servomechanisms with the aim to achieve accurate control and high-performance operation in industrial robots and automated production systems. The inherent complexity and nonlinear behavior of these modules, usually consisting of a servomotor and a speed reducer, often challenge traditional analytical modeling approaches. In response, the study extensively explores the design and implementation of Machine Learning (ML) algorithms to obtain a comprehensive model of the Transmission Error (TE) in rotating vector reducers, which is a main source of robot motion accuracy errors. The ML models are trained with experimental data obtained from a special purpose test rig, where the reducer is tested under different combinations of input speed, applied load and oil temperature. In the second part of the work, the resulting predictive model, tailored to capture the intricate dynamics of the analyzed reducer, is imported into a programmable logic controller to enable online compensation strategies during the execution of custom motion profiles. Experimental tests are conducted using two distinct motion profiles: one generated with a cycloidal law, typical of industrial machinery, and the other extrapolated from the joints of an industrial robot during a pick-and-place task. The results demonstrate the effectiveness of the proposed approach, enabling accurate prediction and substantial reductions (over 90%) in the overall reducer TE through the implemented predictive model.

The future manufacturing system must be capable of supporting customized mass production while reducing cost and must be flexible enough to accommodate market demands. Additionally, workers must possess the knowledge and skills to adapt to the evolving manufacturing environment. Previous studies have been conducted to provide customized manufacturing information to the worker. However, most have not considered the worker's situation or region of interest (ROI), so they had difficulty providing information tailored to the worker. Thus, a manufacturing information recommendation system should utilize not only manufacturing data but also the worker's situational information and intent to assist the worker in adjusting to the evolving working environment. This study presents a smart and user-centric manufacturing information recommendation system that harnesses the vision and text dual encoder-based multimodal deep learning model to offer the most relevant information based on the worker's vision and query, which can support human-robot collaboration (HRC) in a mixed reality (MR) environment. The proposed recommendation model can assist the worker by analyzing the manufacturing environment image acquired from smart glasses, the worker's specific question, and the related manufacturing document. By establishing correlations between the MR-based visual information and the worker's query using the multimodal deep learning model, the proposed approach identifies the most suitable information to be recommended. Furthermore, the recommended information can be visualized through MR smart glasses to support HRC. For quantitative and qualitative evaluation, we compared the proposed model with existing vision-text dual models, and the results demonstrated that the proposed approach outperformed previous studies. Thus, the proposed approach has the potential to assist workers more effectively in MR-based manufacturing environments, enhancing their overall productivity and adaptability.

With the development of intelligent manufacturing, robots are being increasingly applied in manufacturing systems due to their high flexibility. To avoid production disruptions caused by robot failures, higher requirements are imposed on the resilience of systems, specifically in terms of resistance, response, and recovery capabilities. In response to this, this paper investigates the resilient scheduling framework for multi-robot multi-station welding flow shop, thereby endowing and enhancing the resilience of the system. Within the resilient scheduling framework, a proactive scheduling method maximizing resistance capability is firstly proposed based on an improved NSGA-III with variable neighborhood search. Secondly, to improve the response and recovery capabilities of the system, a recovery scheduling method is presented. Therein, an adaptive trigger policy based on deep reinforcement learning is introduced to enhance the rapid response capability for disturbances, while the recovery optimization grants the system the ability to recover its performance that has been degraded due to the impact of disturbances. Finally, through simulation experiments and case study, it is verified that the proposed algorithms and framework possess superior performance of multi-objective optimization, which can endow the multi-robot multi-station welding flow shop with resilience to against uncertain robot failures.

Hand-eye calibration is a prerequisite for robot vision system applications. However, due to the lack of image features, implementing hand-eye calibration with a line-structured light sensor is limited by complex procedures and special reference objects. The aim of this paper is to develop a stable and undemanding eye-in-hand calibration method with a 2D laser sensor that can be adapted to random measurement strategies in most common scenes. The proposed method can use an easily accessible plane from machined workpieces, except for specialized calibration objects, which facilitates the automation of industrial robots. Moreover, in this method, a two-step iterative method is combined with fast simulated annealing based on a single planar constraint to overcome large initial deviations and sensor errors. Simulations and calibration experiments are conducted to assess the method performance and demonstrate the feasibility and accuracy of the proposed eye-in-hand calibration method.

In the context of Industry 4.0, ensuring the compatibility of digital twins (DTs) with existing software systems in the manufacturing sector presents a significant challenge. The Asset Administration Shell (AAS), conceptualized as the standardized DT for an asset, offers a powerful framework that connects the DT with the established software infrastructure through interoperable knowledge representation. Although the IEC 63278 series specifies the AAS metamodel, it lacks a matching strategy for automating the mapping between proprietary data from existing software and AAS information models. Addressing this gap, we introduce a novel dual data mapping system (DDMS) that utilizes a fine-tuned open-source large language model (LLM) for entity matching. This system facilitates not only the mapping between existing software and AAS models but also between AAS models and standardized vocabulary dictionaries, thereby enhancing the model's semantic interoperability. A case study within the injection molding domain illustrates the practical application of DDMS for the automated creation of AAS instances, seamlessly integrating the manufacturer's existing data. Furthermore, we extensively investigate the potential of fine-tuning decode-only LLMs as generative classifiers and encoding-based classifiers for the entity matching task. To this end, we establish two AAS-specific datasets by collecting and compiling AAS-related resources. In addition, supplementary experiments are performed on general entity-matching benchmark datasets to ensure that our empirical conclusions and insights are generally applicable. The experiment results indicate that the fine-tuned generative LLM classifier achieves slightly better results, while the encoding-based classifier enables much faster inference. Furthermore, the fine-tuned LLM surpasses all state-of-the-art approaches for entity matching, including GPT-4 enhanced with in-context learning and chain of thoughts. This evidence highlights the effectiveness of the proposed DDMS in bridging the interoperability gap within DT applications, offering a scalable solution for the manufacturing industry.