Positioning of mango picking point using an improved YOLOv8 architecture with object detection and instance segmentation

IF 4.4 1区 农林科学 Q1 AGRICULTURAL ENGINEERING Biosystems Engineering Pub Date : 2024-09-23 DOI:10.1016/j.biosystemseng.2024.09.015
Hongwei Li , Jianzhi Huang , Zenan Gu , Deqiang He , Junduan Huang , Chenglin Wang
{"title":"Positioning of mango picking point using an improved YOLOv8 architecture with object detection and instance segmentation","authors":"Hongwei Li ,&nbsp;Jianzhi Huang ,&nbsp;Zenan Gu ,&nbsp;Deqiang He ,&nbsp;Junduan Huang ,&nbsp;Chenglin Wang","doi":"10.1016/j.biosystemseng.2024.09.015","DOIUrl":null,"url":null,"abstract":"<div><div>Positioning of mango picking points is a crucial technology for the realisation of automated robotic mango harvesting. Herein, this study reported a visualised end-to-end system for mango picking point positioning using improved YOLOv8 architecture with object detection and instance segmentation, as well as an algorithm of picking point positioning. At first, the improved YOLOv8n model, incorporating the BiFPN structure and the SPD-Conv module, was utilised to enhance the detection performance of mango fruits and stems. This model achieved a detection precision of 98.9% in fruits and 97.1% in stems, with recall of 99.5% and 94.6% respectively. Then, the YOLOv8n-seg model was used for segment the stem ROI (Region of interest), leading to 81.85% in MIoU and 88.69% in mPA. Finally, a skeleton line of the stem region was obtained on the basis of the segmentation image, and a picking point positioning algorithm was developed to determine the coordinates of the optimal picking point. Subsequently, the positioning success rate of coordinates, absolute errors, and relative errors were calculated by comparing the automatic positioned coordinates with the manually positioned stem region. Experimental results indicated that this study achieved an average positioning success rate of 92.01%, with an average absolute error of 4.93 pixels and an average relative error of 13.11%. Additionally, the average processing time for processing 640 images using the picking point positioning system is 72.75 ms. This study demonstrates the reliability and effectiveness of positioning mango picking points, laying the technological basis for the automated harvesting of mango fruits.</div></div>","PeriodicalId":9173,"journal":{"name":"Biosystems Engineering","volume":"247 ","pages":"Pages 202-220"},"PeriodicalIF":4.4000,"publicationDate":"2024-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Biosystems Engineering","FirstCategoryId":"97","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1537511024002174","RegionNum":1,"RegionCategory":"农林科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"AGRICULTURAL ENGINEERING","Score":null,"Total":0}
引用次数: 0

Abstract

Positioning of mango picking points is a crucial technology for the realisation of automated robotic mango harvesting. Herein, this study reported a visualised end-to-end system for mango picking point positioning using improved YOLOv8 architecture with object detection and instance segmentation, as well as an algorithm of picking point positioning. At first, the improved YOLOv8n model, incorporating the BiFPN structure and the SPD-Conv module, was utilised to enhance the detection performance of mango fruits and stems. This model achieved a detection precision of 98.9% in fruits and 97.1% in stems, with recall of 99.5% and 94.6% respectively. Then, the YOLOv8n-seg model was used for segment the stem ROI (Region of interest), leading to 81.85% in MIoU and 88.69% in mPA. Finally, a skeleton line of the stem region was obtained on the basis of the segmentation image, and a picking point positioning algorithm was developed to determine the coordinates of the optimal picking point. Subsequently, the positioning success rate of coordinates, absolute errors, and relative errors were calculated by comparing the automatic positioned coordinates with the manually positioned stem region. Experimental results indicated that this study achieved an average positioning success rate of 92.01%, with an average absolute error of 4.93 pixels and an average relative error of 13.11%. Additionally, the average processing time for processing 640 images using the picking point positioning system is 72.75 ms. This study demonstrates the reliability and effectiveness of positioning mango picking points, laying the technological basis for the automated harvesting of mango fruits.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
利用改进的 YOLOv8 架构进行芒果采摘点定位,并进行对象检测和实例分割
芒果采摘点的定位是实现芒果自动机器人采摘的关键技术。在此,本研究报告了一个可视化端到端芒果采摘点定位系统,该系统采用改进的 YOLOv8 架构,具有对象检测和实例分割功能,以及采摘点定位算法。首先,利用改进的 YOLOv8n 模型,结合 BiFPN 结构和 SPD-Conv 模块,提高了芒果果实和茎的检测性能。该模型对水果和茎的检测精度分别达到了 98.9% 和 97.1%,召回率分别为 99.5% 和 94.6%。然后,使用 YOLOv8n-seg 模型对茎的 ROI(感兴趣区域)进行分割,MIoU 和 mPA 的分割结果分别为 81.85% 和 88.69%。最后,在分割图像的基础上获得了茎干区域的骨架线,并开发了取点定位算法,以确定最佳取点的坐标。随后,通过比较自动定位的坐标和人工定位的茎干区域,计算出坐标的定位成功率、绝对误差和相对误差。实验结果表明,该研究的平均定位成功率为 92.01%,平均绝对误差为 4.93 像素,平均相对误差为 13.11%。此外,使用拾取点定位系统处理 640 幅图像的平均处理时间为 72.75 毫秒。这项研究证明了芒果采摘点定位的可靠性和有效性,为芒果果实的自动采摘奠定了技术基础。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
Biosystems Engineering
Biosystems Engineering 农林科学-农业工程
CiteScore
10.60
自引率
7.80%
发文量
239
审稿时长
53 days
期刊介绍: Biosystems Engineering publishes research in engineering and the physical sciences that represent advances in understanding or modelling of the performance of biological systems for sustainable developments in land use and the environment, agriculture and amenity, bioproduction processes and the food chain. The subject matter of the journal reflects the wide range and interdisciplinary nature of research in engineering for biological systems.
期刊最新文献
Egg characteristics assessment as an enabler for in-ovo sexing technology: A review Analysis of three-dimensional cake thickness structure characteristics in a screen filter for drip irrigation based on the CFD‒DEM coupling method Stiffness evaluation of semi-rigid connection using steel clamps in plastic greenhouse structure Vacuum suction end-effector development for robotic harvesters of fresh market apples Optimisation design and experimental analysis of rotary blade reinforcing ribs using DEM-FEM techniques
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1