Manipulation Skill Representation and Knowledge Reasoning for 3C Assembly

Abstract

Traditional manufacturing in the computer, communication, and consumer electronics (3C) industries primarily relies on automation but lacks autonomous learning, decision-making, and adaptability. To address this challenge, this study introduces a multi-layer multi-level knowledge representation (MLMLKR) approach aimed at enhancing the adaptability and accuracy of assembly processes in 3C tasks. The MLMLKR framework comprises five layers: task, area, object, action, and agent, as well as three levels: ontology, template, and instance. In addition, we have developed a comprehensive set of templates, including master templates, task subtemplates, and action subtemplates, to improve the transferability of robotic operations in 3C scenarios. To realize knowledge-enhanced task-to-action reasoning (KET2A), we utilize the learnable composite loss function to combine BERT module, graph attention network module and Seq2Seq module into a unified end-to-end model. Finally, we employ multilayer multi-level knowledge graph and KET2A into multiple assembly scenarios, further validate the capabilities of task transfer and action transfer. The experiment results demonstrated that the action sequence reasoning accuracy reached to 98.1% and the perplexity was 1.019. The reasoned sequence execution of subscriber identity module and memory module assembly tasks were performed in real scenarios. Furthermore, the proposed scheme accomplished congeneric object operating transfer from DDR3 8 G 138 × 38.8 mm to double data rate 3 (DDR3) 4 G 138 × 18.8 mm in memory module assembly task, also from Intel Core CPU to advanced micro devices (AMD) CPU in CPU assembly task. Meanwhile, it implemented congeneric action “press” operating transfer between CPU, memory module, and front camera assembly tasks. In a word, the proposed manipulation skill representation and knowledge reasoning method is feasible and can be applied in intelligent 3C assembly.