Vision-based food handling system for high-resemblance random food items

IF 1.9 4区 计算机科学 Q3 ROBOTICS Robotica Pub Date : 2024-02-21 DOI:10.1017/s0263574724000122
Yadan Zeng, Yee Seng Teoh, Guoniu Zhu, Elvin Toh, I-Ming Chen
{"title":"Vision-based food handling system for high-resemblance random food items","authors":"Yadan Zeng, Yee Seng Teoh, Guoniu Zhu, Elvin Toh, I-Ming Chen","doi":"10.1017/s0263574724000122","DOIUrl":null,"url":null,"abstract":"The rise in the number of automated robotic kitchens accelerated the need for advanced food handling system, emphasizing food analysis including ingredient classification pose recognition and assembling strategy. Selecting the optimal piece from a pile of similarly shaped food items is a challenge to automated meal assembling system. To address this, we present a constructive assembling algorithm, introducing a unique approach for food pose detection–Fast Image to Pose Detection (FI2PD), and a closed-loop packing strategy. Powered by a convolutional neural network (CNN) and a pose retrieval model, FI2PD is adept at constructing a 6D pose from only RGB images. The method employs a coarse-to-fine approach, leveraging the CNN to pinpoint object orientation and position, alongside a pose retrieval process for target selection and 6D pose derivation. Our closed-loop packing strategy, aided by the Item Arrangement Verifier, ensures precise arrangement and system robustness. Additionally, we introduce our <jats:italic>FdIngred328</jats:italic> dataset of nine food categories ranging from fake foods to real foods, and the automatically generated data based on synthetic techniques. The performance of our method for object recognition and pose detection has been demonstrated to achieve a success rate of 97.9%. Impressively, the integration of a closed-loop strategy into our meal-assembly process resulted in a notable success rate of 90%, outperforming the results of systems lacking the closed-loop mechanism.","PeriodicalId":49593,"journal":{"name":"Robotica","volume":null,"pages":null},"PeriodicalIF":1.9000,"publicationDate":"2024-02-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Robotica","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1017/s0263574724000122","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"ROBOTICS","Score":null,"Total":0}
引用次数: 0

Abstract

The rise in the number of automated robotic kitchens accelerated the need for advanced food handling system, emphasizing food analysis including ingredient classification pose recognition and assembling strategy. Selecting the optimal piece from a pile of similarly shaped food items is a challenge to automated meal assembling system. To address this, we present a constructive assembling algorithm, introducing a unique approach for food pose detection–Fast Image to Pose Detection (FI2PD), and a closed-loop packing strategy. Powered by a convolutional neural network (CNN) and a pose retrieval model, FI2PD is adept at constructing a 6D pose from only RGB images. The method employs a coarse-to-fine approach, leveraging the CNN to pinpoint object orientation and position, alongside a pose retrieval process for target selection and 6D pose derivation. Our closed-loop packing strategy, aided by the Item Arrangement Verifier, ensures precise arrangement and system robustness. Additionally, we introduce our FdIngred328 dataset of nine food categories ranging from fake foods to real foods, and the automatically generated data based on synthetic techniques. The performance of our method for object recognition and pose detection has been demonstrated to achieve a success rate of 97.9%. Impressively, the integration of a closed-loop strategy into our meal-assembly process resulted in a notable success rate of 90%, outperforming the results of systems lacking the closed-loop mechanism.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
基于视觉的食品处理系统,用于处理高重现性随机食品
自动化机器人厨房数量的增加加速了对先进食品处理系统的需求,强调食品分析,包括配料分类姿势识别和装配策略。从一堆形状相似的食品中挑选出最佳部件是自动配餐系统面临的一项挑战。为此,我们提出了一种建设性的组装算法,引入了一种独特的食品姿态检测方法--快速图像姿态检测(FI2PD)和一种闭环包装策略。在卷积神经网络(CNN)和姿势检索模型的支持下,FI2PD 擅长从仅有的 RGB 图像中构建 6D 姿势。该方法采用了从粗到细的方法,利用卷积神经网络精确定位物体的方向和位置,同时利用姿态检索过程进行目标选择和 6D 姿态推导。我们的闭环包装策略在 "物品排列验证器 "的辅助下,确保了精确的排列和系统的鲁棒性。此外,我们还介绍了我们的 FdIngred328 数据集,该数据集包含从假冒食品到真实食品的九个食品类别,以及基于合成技术自动生成的数据。结果表明,我们的方法在物体识别和姿势检测方面的成功率高达 97.9%。令人印象深刻的是,将闭环策略整合到我们的膳食组装流程中后,成功率达到了 90%,明显优于缺乏闭环机制的系统。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
Robotica
Robotica 工程技术-机器人学
CiteScore
4.50
自引率
22.20%
发文量
181
审稿时长
9.9 months
期刊介绍: Robotica is a forum for the multidisciplinary subject of robotics and encourages developments, applications and research in this important field of automation and robotics with regard to industry, health, education and economic and social aspects of relevance. Coverage includes activities in hostile environments, applications in the service and manufacturing industries, biological robotics, dynamics and kinematics involved in robot design and uses, on-line robots, robot task planning, rehabilitation robotics, sensory perception, software in the widest sense, particularly in respect of programming languages and links with CAD/CAM systems, telerobotics and various other areas. In addition, interest is focused on various Artificial Intelligence topics of theoretical and practical interest.
期刊最新文献
Combining spatial clustering and tour planning for efficient full area exploration Design of a robotic gripper for casting sorting robots with rigid–flexible coupling structures DDPG-based path planning for cable-driven manipulators in multi-obstacle environments An online payload identification method based on parameter difference for industrial robots Adaptive integral terminal sliding mode control of unmanned bicycle via ELM and barrier function
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1