Intelligent sorting method for assembly line based on visual positioning and model predictive control of robotic arm

IF 2.2 2区 农林科学 Q2 AGRICULTURAL ENGINEERING International Journal of Agricultural and Biological Engineering Pub Date : 2023-01-01 DOI:10.25165/j.ijabe.20231604.7908
Ruining Zhang, Wei Lu, Xingliang Jian, Hui Luo
{"title":"Intelligent sorting method for assembly line based on visual positioning and model predictive control of robotic arm","authors":"Ruining Zhang, Wei Lu, Xingliang Jian, Hui Luo","doi":"10.25165/j.ijabe.20231604.7908","DOIUrl":null,"url":null,"abstract":"The existing steering device in the fruit and vegetable packaging assembly line cannot adjust the attitude of lettuce to a unified attitude, affecting the input and packaging process of the packaging machine. This study proposes an intelligent assembly line sorting method based on the visual positioning and model predictive control of a robotic arm. First, lightweight improvement based on the YOLOv5 is realized, the lettuce stalk in the background of the conveyor belt is promptly identified, the image of the lettuce stalk in the anchor box area is processed, and the edge contour point set is determined to extract the pixel coordinates of the optimal grasp point and mirror inclination angle of the lettuce. For the intelligent assembly line system, a robot arm kinematics model is constructed and the robot kinematics inverse solutions are calculated. Additionally, the lettuce movement speeds are dynamically measured by the vision system. A combination of the model prediction control, dynamic tracking, and rapid sorting of the lettuce by the robot claw is realized. The results show that the average detection time of a single frame image in the visual positioning part is 0.014 s, which is reduced by 50%; the accuracy and recall are 98% and 95%, respectively. The detection time is significantly reduced by ensuring accuracy. Within the current speed range of the packaging assembly line conveyor belt, the manipulator can grasp lettuce at different speeds stably and fast; the average axial error, average radial error, and adjusted average inclination angle error are 0.71 cm, 1.02 cm, and 3.79°, respectively, verifying the high efficiency and stability of the model. The proposed method of this study enables application in the intelligent sorting operation of industrial assembly lines Keywords: YOLOv5, deep learning, image recognition, model predictive control, intelligent assembly line DOI: 10.25165/j.ijabe.20231604.7908 Citation: Zhang R N, Lu W, Jian X L, Luo H. Intelligent sorting method for assembly line based on visual positioning and model predictive control of robotic arm. Int J Agric & Biol Eng, 2023; 16(4): 207-214.","PeriodicalId":13895,"journal":{"name":"International Journal of Agricultural and Biological Engineering","volume":null,"pages":null},"PeriodicalIF":2.2000,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Agricultural and Biological Engineering","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.25165/j.ijabe.20231604.7908","RegionNum":2,"RegionCategory":"农林科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"AGRICULTURAL ENGINEERING","Score":null,"Total":0}
引用次数: 0

Abstract

The existing steering device in the fruit and vegetable packaging assembly line cannot adjust the attitude of lettuce to a unified attitude, affecting the input and packaging process of the packaging machine. This study proposes an intelligent assembly line sorting method based on the visual positioning and model predictive control of a robotic arm. First, lightweight improvement based on the YOLOv5 is realized, the lettuce stalk in the background of the conveyor belt is promptly identified, the image of the lettuce stalk in the anchor box area is processed, and the edge contour point set is determined to extract the pixel coordinates of the optimal grasp point and mirror inclination angle of the lettuce. For the intelligent assembly line system, a robot arm kinematics model is constructed and the robot kinematics inverse solutions are calculated. Additionally, the lettuce movement speeds are dynamically measured by the vision system. A combination of the model prediction control, dynamic tracking, and rapid sorting of the lettuce by the robot claw is realized. The results show that the average detection time of a single frame image in the visual positioning part is 0.014 s, which is reduced by 50%; the accuracy and recall are 98% and 95%, respectively. The detection time is significantly reduced by ensuring accuracy. Within the current speed range of the packaging assembly line conveyor belt, the manipulator can grasp lettuce at different speeds stably and fast; the average axial error, average radial error, and adjusted average inclination angle error are 0.71 cm, 1.02 cm, and 3.79°, respectively, verifying the high efficiency and stability of the model. The proposed method of this study enables application in the intelligent sorting operation of industrial assembly lines Keywords: YOLOv5, deep learning, image recognition, model predictive control, intelligent assembly line DOI: 10.25165/j.ijabe.20231604.7908 Citation: Zhang R N, Lu W, Jian X L, Luo H. Intelligent sorting method for assembly line based on visual positioning and model predictive control of robotic arm. Int J Agric & Biol Eng, 2023; 16(4): 207-214.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
基于机械臂视觉定位和模型预测控制的装配线智能分拣方法
果蔬包装装配线中现有的转向装置无法将生菜的姿态调整到统一的姿态,影响包装机的输入和包装过程。提出了一种基于机械臂视觉定位和模型预测控制的智能装配线分拣方法。首先,实现基于YOLOv5的轻量化改进,及时识别输送带背景中的生菜茎秆,对锚箱区域的生菜茎秆图像进行处理,确定边缘轮廓点集,提取生菜最优抓取点和镜像倾角的像素坐标。针对智能装配线系统,建立了机器人手臂的运动学模型,计算了机器人的运动学逆解。此外,生菜的运动速度由视觉系统动态测量。实现了模型预测控制、动态跟踪和机器人爪对生菜的快速分拣相结合。结果表明:视觉定位部分单帧图像的平均检测时间为0.014 s,降低了50%;准确率和召回率分别为98%和95%。通过确保准确性,大大缩短了检测时间。在目前包装装配线输送带的速度范围内,机械手可以稳定快速地抓取不同速度的生菜;平均轴向误差、平均径向误差和调整后的平均倾角误差分别为0.71 cm、1.02 cm和3.79°,验证了模型的高效率和稳定性。关键词:YOLOv5,深度学习,图像识别,模型预测控制,智能装配线DOI: 10.25165/ j.j ijabe.20231604.7908引用本文:张瑞宁,卢伟,简晓玲,罗慧。基于视觉定位和机械臂模型预测控制的装配线智能分拣方法农业与生物工程学报,2023;16(4): 207 - 214。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
CiteScore
4.30
自引率
12.50%
发文量
88
审稿时长
24 weeks
期刊介绍: International Journal of Agricultural and Biological Engineering (IJABE, https://www.ijabe.org) is a peer reviewed open access international journal. IJABE, started in 2008, is a joint publication co-sponsored by US-based Association of Agricultural, Biological and Food Engineers (AOCABFE) and China-based Chinese Society of Agricultural Engineering (CSAE). The ISSN 1934-6344 and eISSN 1934-6352 numbers for both print and online IJABE have been registered in US. Now, Int. J. Agric. & Biol. Eng (IJABE) is published in both online and print version by Chinese Academy of Agricultural Engineering.
期刊最新文献
Effects of spray adjuvants and operation modes on droplet deposition and elm aphid control using an unmanned aerial vehicle Comprehensive evaluation of the optimal rates of irrigation and potassium application for strawberry Design and test of the bilateral throwing soil-covering device for straw mulching machine in orchards Detection of the foreign object positions in agricultural soils using Mask-RCNN Separation and mechanical properties of residual film and soil
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1