Hao Li, Gengdai Liu, Haoqi Wang, Xiaoyu Wen, G. Xie, Guofu Luo, Shuai Zhang, Miying Yang
{"title":"Mechanical movement data acquisition method based on the multilayer neural networks and machine vision in a digital twin environment","authors":"Hao Li, Gengdai Liu, Haoqi Wang, Xiaoyu Wen, G. Xie, Guofu Luo, Shuai Zhang, Miying Yang","doi":"10.12688/digitaltwin.17441.1","DOIUrl":null,"url":null,"abstract":"Background: Digital twin requires virtual reality mapping and optimization iteration between physical devices and virtual models. The mechanical movement data collection of physical equipment is essential for the implementation of accurate virtual and physical synchronization in a digital twin environment. However, the traditional approach relying on PLC (programmable logic control) fails to collect various mechanical motion state data. Additionally, few investigations have used machine visions for the virtual and physical synchronization of equipment. Thus, this paper presents a mechanical movement data acquisition method based on multilayer neural networks and machine vision. Methods: Firstly, various visual marks with different colors and shapes are designed for marking physical devices. Secondly, a recognition method based on the Hough transform and histogram feature is proposed to realize the recognition of shape and color features respectively. Then, the multilayer neural network model is introduced in the visual mark location. The neural network is trained by the dropout algorithm to realize the tracking and location of the visual mark. To test the proposed method, 1000 samples were selected. Results: The experiment results shows that when the size of the visual mark is larger than 6mm, the recognition success rate of the recognition algorithm can reach more than 95%. In the actual operation environment with multiple cameras, the identification points can be located more accurately. Moreover, the camera calibration process of binocular and multi-eye vision can be simplified by the multilayer neural networks. Conclusions: This study proposes an effective method in the collection of mechanical motion data of physical equipment in a digital twin environment. Further studies are needed to perceive posture and shape data of physical entities under the multi-camera redundant shooting.","PeriodicalId":29831,"journal":{"name":"Digital Twin","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2021-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"2","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Digital Twin","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.12688/digitaltwin.17441.1","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 2
Abstract
Background: Digital twin requires virtual reality mapping and optimization iteration between physical devices and virtual models. The mechanical movement data collection of physical equipment is essential for the implementation of accurate virtual and physical synchronization in a digital twin environment. However, the traditional approach relying on PLC (programmable logic control) fails to collect various mechanical motion state data. Additionally, few investigations have used machine visions for the virtual and physical synchronization of equipment. Thus, this paper presents a mechanical movement data acquisition method based on multilayer neural networks and machine vision. Methods: Firstly, various visual marks with different colors and shapes are designed for marking physical devices. Secondly, a recognition method based on the Hough transform and histogram feature is proposed to realize the recognition of shape and color features respectively. Then, the multilayer neural network model is introduced in the visual mark location. The neural network is trained by the dropout algorithm to realize the tracking and location of the visual mark. To test the proposed method, 1000 samples were selected. Results: The experiment results shows that when the size of the visual mark is larger than 6mm, the recognition success rate of the recognition algorithm can reach more than 95%. In the actual operation environment with multiple cameras, the identification points can be located more accurately. Moreover, the camera calibration process of binocular and multi-eye vision can be simplified by the multilayer neural networks. Conclusions: This study proposes an effective method in the collection of mechanical motion data of physical equipment in a digital twin environment. Further studies are needed to perceive posture and shape data of physical entities under the multi-camera redundant shooting.
期刊介绍:
Digital Twin is a rapid multidisciplinary open access publishing platform for state-of-the-art, basic, scientific and applied research on digital twin technologies. Digital Twin covers all areas related digital twin technologies, including broad fields such as smart manufacturing, civil and industrial engineering, healthcare, agriculture, and many others. The platform is open to submissions from researchers, practitioners and experts, and all articles will benefit from open peer review.
The aim of Digital Twin is to advance the state-of-the-art in digital twin research and encourage innovation by highlighting efficient, robust and sustainable multidisciplinary applications across a variety of fields. Challenges can be addressed using theoretical, methodological, and technological approaches.
The scope of Digital Twin includes, but is not limited to, the following areas:
● Digital twin concepts, architecture, and frameworks
● Digital twin theory and method
● Digital twin key technologies and tools
● Digital twin applications and case studies
● Digital twin implementation
● Digital twin services
● Digital twin security
● Digital twin standards
Digital twin also focuses on applications within and across broad sectors including:
● Smart manufacturing
● Aviation and aerospace
● Smart cities and construction
● Healthcare and medicine
● Robotics
● Shipping, vehicles and railways
● Industrial engineering and engineering management
● Agriculture
● Mining
● Power, energy and environment
Digital Twin features a range of article types including research articles, case studies, method articles, study protocols, software tools, systematic reviews, data notes, brief reports, and opinion articles.