Comparison of lidar semantic segmentation performance on the structured SemanticKITTI and off-road RELLIS-3D datasets

Mason McVicker, Lauren Ervin, Yongzhi Yang, Kenneth G. Ricks
{"title":"Comparison of lidar semantic segmentation performance on the structured SemanticKITTI and off-road RELLIS-3D datasets","authors":"Mason McVicker, Lauren Ervin, Yongzhi Yang, Kenneth G. Ricks","doi":"10.1007/s41315-024-00376-5","DOIUrl":null,"url":null,"abstract":"<p>Existing lidar-based semantic segmentation algorithms and datasets focus on autonomous vehicles operating in urban environments. This has greatly improved the safety and reliability of these autonomous vehicles in predictable scenery. A new dataset provides lidar data focusing on off-road environments as seen by autonomous ground vehicles, ushering in a new era of off-road exploration capabilities. To the best of our knowledge, no new algorithms have been developed specifically for this unstructured environment. To gain an understanding of how existing algorithms perform in an off-road environment, we assess the baseline performance of four algorithms, KPConv, SalsaNext, Cylinder3D, and SphereFormer, on a commonly used on-road dataset, SemanticKITTI. We then compare the results with an off-road dataset, RELLIS-3D. We discuss the degradation of each algorithm on the off-road dataset and investigate potential causes such as class imbalance, inconsistencies in the labeled data, and the inherent difficulty of segmenting off-road environments. We present the strengths and weaknesses of each algorithm’s segmentation abilities and provide a comparison of the runtime of each algorithm for real-time capabilities. This is crucial for identifying what network architecture features are potentially the most beneficial for unstructured scenes. A robust, open-source software implementation via docker containers and bash scripts provides simple, repeatable execution of all algorithm training and evaluations. All code is publicly available at https://github.com/UA-Lidar-Segmentation-Research.</p>","PeriodicalId":44563,"journal":{"name":"International Journal of Intelligent Robotics and Applications","volume":"158 1","pages":""},"PeriodicalIF":2.1000,"publicationDate":"2024-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Intelligent Robotics and Applications","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1007/s41315-024-00376-5","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"ROBOTICS","Score":null,"Total":0}
引用次数: 0

Abstract

Existing lidar-based semantic segmentation algorithms and datasets focus on autonomous vehicles operating in urban environments. This has greatly improved the safety and reliability of these autonomous vehicles in predictable scenery. A new dataset provides lidar data focusing on off-road environments as seen by autonomous ground vehicles, ushering in a new era of off-road exploration capabilities. To the best of our knowledge, no new algorithms have been developed specifically for this unstructured environment. To gain an understanding of how existing algorithms perform in an off-road environment, we assess the baseline performance of four algorithms, KPConv, SalsaNext, Cylinder3D, and SphereFormer, on a commonly used on-road dataset, SemanticKITTI. We then compare the results with an off-road dataset, RELLIS-3D. We discuss the degradation of each algorithm on the off-road dataset and investigate potential causes such as class imbalance, inconsistencies in the labeled data, and the inherent difficulty of segmenting off-road environments. We present the strengths and weaknesses of each algorithm’s segmentation abilities and provide a comparison of the runtime of each algorithm for real-time capabilities. This is crucial for identifying what network architecture features are potentially the most beneficial for unstructured scenes. A robust, open-source software implementation via docker containers and bash scripts provides simple, repeatable execution of all algorithm training and evaluations. All code is publicly available at https://github.com/UA-Lidar-Segmentation-Research.

Abstract Image

查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
结构化 SemanticKITTI 和越野 RELLIS-3D 数据集上的激光雷达语义分割性能比较
现有的基于激光雷达的语义分割算法和数据集主要针对在城市环境中运行的自动驾驶车辆。这大大提高了这些自动驾驶车辆在可预测场景中的安全性和可靠性。一个新的数据集提供的激光雷达数据侧重于自动驾驶地面车辆看到的越野环境,开创了越野探索能力的新时代。据我们所知,还没有专门针对这种非结构化环境开发的新算法。为了了解现有算法在非道路环境中的表现,我们在常用的道路数据集 SemanticKITTI 上评估了 KPConv、SalsaNext、Cylinder3D 和 SphereFormer 四种算法的基准性能。然后,我们将结果与非道路数据集 RELLIS-3D 进行比较。我们讨论了每种算法在非道路数据集上的性能下降情况,并研究了潜在的原因,如类不平衡、标注数据的不一致性以及分割非道路环境的固有难度。我们介绍了每种算法分割能力的优缺点,并比较了每种算法的实时运行时间。这对于确定哪些网络架构功能可能对非结构化场景最有利至关重要。通过 docker 容器和 bash 脚本实现的强大开源软件可简单、可重复地执行所有算法的训练和评估。所有代码均可在 https://github.com/UA-Lidar-Segmentation-Research 上公开获取。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
CiteScore
3.80
自引率
5.90%
发文量
50
期刊介绍: The International Journal of Intelligent Robotics and Applications (IJIRA) fosters the dissemination of new discoveries and novel technologies that advance developments in robotics and their broad applications. This journal provides a publication and communication platform for all robotics topics, from the theoretical fundamentals and technological advances to various applications including manufacturing, space vehicles, biomedical systems and automobiles, data-storage devices, healthcare systems, home appliances, and intelligent highways. IJIRA welcomes contributions from researchers, professionals and industrial practitioners. It publishes original, high-quality and previously unpublished research papers, brief reports, and critical reviews. Specific areas of interest include, but are not limited to:Advanced actuators and sensorsCollective and social robots Computing, communication and controlDesign, modeling and prototypingHuman and robot interactionMachine learning and intelligenceMobile robots and intelligent autonomous systemsMulti-sensor fusion and perceptionPlanning, navigation and localizationRobot intelligence, learning and linguisticsRobotic vision, recognition and reconstructionBio-mechatronics and roboticsCloud and Swarm roboticsCognitive and neuro roboticsExploration and security roboticsHealthcare, medical and assistive roboticsRobotics for intelligent manufacturingService, social and entertainment roboticsSpace and underwater robotsNovel and emerging applications
期刊最新文献
A review of the application of fuzzy mathematical algorithm-based approach in autonomous vehicles and drones Robotic tree climbers and strategies - a survey Efficient multi-robot path planning in real environments: a centralized coordination system Cross-pollination of knowledge for object detection in domain adaptation for industrial automation Push or pull: grasping performance analysis between a pulling gripper inspired by Tetraonchus monenteron parasite versus an actively pushing gripper developed through many-objective design optimization
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1