{"title":"Intelligent sorting method for assembly line based on visual positioning and model predictive control of robotic arm","authors":"Ruining Zhang, Wei Lu, Xingliang Jian, Hui Luo","doi":"10.25165/j.ijabe.20231604.7908","DOIUrl":null,"url":null,"abstract":"The existing steering device in the fruit and vegetable packaging assembly line cannot adjust the attitude of lettuce to a unified attitude, affecting the input and packaging process of the packaging machine. This study proposes an intelligent assembly line sorting method based on the visual positioning and model predictive control of a robotic arm. First, lightweight improvement based on the YOLOv5 is realized, the lettuce stalk in the background of the conveyor belt is promptly identified, the image of the lettuce stalk in the anchor box area is processed, and the edge contour point set is determined to extract the pixel coordinates of the optimal grasp point and mirror inclination angle of the lettuce. For the intelligent assembly line system, a robot arm kinematics model is constructed and the robot kinematics inverse solutions are calculated. Additionally, the lettuce movement speeds are dynamically measured by the vision system. A combination of the model prediction control, dynamic tracking, and rapid sorting of the lettuce by the robot claw is realized. The results show that the average detection time of a single frame image in the visual positioning part is 0.014 s, which is reduced by 50%; the accuracy and recall are 98% and 95%, respectively. The detection time is significantly reduced by ensuring accuracy. Within the current speed range of the packaging assembly line conveyor belt, the manipulator can grasp lettuce at different speeds stably and fast; the average axial error, average radial error, and adjusted average inclination angle error are 0.71 cm, 1.02 cm, and 3.79°, respectively, verifying the high efficiency and stability of the model. The proposed method of this study enables application in the intelligent sorting operation of industrial assembly lines Keywords: YOLOv5, deep learning, image recognition, model predictive control, intelligent assembly line DOI: 10.25165/j.ijabe.20231604.7908 Citation: Zhang R N, Lu W, Jian X L, Luo H. Intelligent sorting method for assembly line based on visual positioning and model predictive control of robotic arm. Int J Agric & Biol Eng, 2023; 16(4): 207-214.","PeriodicalId":13895,"journal":{"name":"International Journal of Agricultural and Biological Engineering","volume":null,"pages":null},"PeriodicalIF":2.2000,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Agricultural and Biological Engineering","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.25165/j.ijabe.20231604.7908","RegionNum":2,"RegionCategory":"农林科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"AGRICULTURAL ENGINEERING","Score":null,"Total":0}
引用次数: 0
Abstract
The existing steering device in the fruit and vegetable packaging assembly line cannot adjust the attitude of lettuce to a unified attitude, affecting the input and packaging process of the packaging machine. This study proposes an intelligent assembly line sorting method based on the visual positioning and model predictive control of a robotic arm. First, lightweight improvement based on the YOLOv5 is realized, the lettuce stalk in the background of the conveyor belt is promptly identified, the image of the lettuce stalk in the anchor box area is processed, and the edge contour point set is determined to extract the pixel coordinates of the optimal grasp point and mirror inclination angle of the lettuce. For the intelligent assembly line system, a robot arm kinematics model is constructed and the robot kinematics inverse solutions are calculated. Additionally, the lettuce movement speeds are dynamically measured by the vision system. A combination of the model prediction control, dynamic tracking, and rapid sorting of the lettuce by the robot claw is realized. The results show that the average detection time of a single frame image in the visual positioning part is 0.014 s, which is reduced by 50%; the accuracy and recall are 98% and 95%, respectively. The detection time is significantly reduced by ensuring accuracy. Within the current speed range of the packaging assembly line conveyor belt, the manipulator can grasp lettuce at different speeds stably and fast; the average axial error, average radial error, and adjusted average inclination angle error are 0.71 cm, 1.02 cm, and 3.79°, respectively, verifying the high efficiency and stability of the model. The proposed method of this study enables application in the intelligent sorting operation of industrial assembly lines Keywords: YOLOv5, deep learning, image recognition, model predictive control, intelligent assembly line DOI: 10.25165/j.ijabe.20231604.7908 Citation: Zhang R N, Lu W, Jian X L, Luo H. Intelligent sorting method for assembly line based on visual positioning and model predictive control of robotic arm. Int J Agric & Biol Eng, 2023; 16(4): 207-214.
期刊介绍:
International Journal of Agricultural and Biological Engineering (IJABE, https://www.ijabe.org) is a peer reviewed open access international journal. IJABE, started in 2008, is a joint publication co-sponsored by US-based Association of Agricultural, Biological and Food Engineers (AOCABFE) and China-based Chinese Society of Agricultural Engineering (CSAE). The ISSN 1934-6344 and eISSN 1934-6352 numbers for both print and online IJABE have been registered in US. Now, Int. J. Agric. & Biol. Eng (IJABE) is published in both online and print version by Chinese Academy of Agricultural Engineering.