WaterSAM:将 SAM 用于水下物体分割

IF 2.7 3区 地球科学 Q1 ENGINEERING, MARINE Journal of Marine Science and Engineering Pub Date : 2024-09-11 DOI:10.3390/jmse12091616
Yang Hong, Xiaowei Zhou, Ruzhuang Hua, Qingxuan Lv, Junyu Dong
{"title":"WaterSAM:将 SAM 用于水下物体分割","authors":"Yang Hong, Xiaowei Zhou, Ruzhuang Hua, Qingxuan Lv, Junyu Dong","doi":"10.3390/jmse12091616","DOIUrl":null,"url":null,"abstract":"Object segmentation, a key type of image segmentation, focuses on detecting and delineating individual objects within an image, essential for applications like robotic vision and augmented reality. Despite advancements in deep learning improving object segmentation, underwater object segmentation remains challenging due to unique underwater complexities such as turbulence diffusion, light absorption, noise, low contrast, uneven illumination, and intricate backgrounds. The scarcity of underwater datasets further complicates these challenges. The Segment Anything Model (SAM) has shown potential in addressing these issues, but its adaptation for underwater environments, AquaSAM, requires fine-tuning all parameters, demanding more labeled data and high computational costs. In this paper, we propose WaterSAM, an adapted model for underwater object segmentation. Inspired by Low-Rank Adaptation (LoRA), WaterSAM incorporates trainable rank decomposition matrices into the Transformer’s layers, specifically enhancing the image encoder. This approach significantly reduces the number of trainable parameters to 6.7% of SAM’s parameters, lowering computational costs. We validated WaterSAM on three underwater image datasets: COD10K, SUIM, and UIIS. Results demonstrate that WaterSAM significantly outperforms pre-trained SAM in underwater segmentation tasks, contributing to advancements in marine biology, underwater archaeology, and environmental monitoring.","PeriodicalId":16168,"journal":{"name":"Journal of Marine Science and Engineering","volume":"-1 1","pages":""},"PeriodicalIF":2.7000,"publicationDate":"2024-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"WaterSAM: Adapting SAM for Underwater Object Segmentation\",\"authors\":\"Yang Hong, Xiaowei Zhou, Ruzhuang Hua, Qingxuan Lv, Junyu Dong\",\"doi\":\"10.3390/jmse12091616\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Object segmentation, a key type of image segmentation, focuses on detecting and delineating individual objects within an image, essential for applications like robotic vision and augmented reality. Despite advancements in deep learning improving object segmentation, underwater object segmentation remains challenging due to unique underwater complexities such as turbulence diffusion, light absorption, noise, low contrast, uneven illumination, and intricate backgrounds. The scarcity of underwater datasets further complicates these challenges. The Segment Anything Model (SAM) has shown potential in addressing these issues, but its adaptation for underwater environments, AquaSAM, requires fine-tuning all parameters, demanding more labeled data and high computational costs. In this paper, we propose WaterSAM, an adapted model for underwater object segmentation. Inspired by Low-Rank Adaptation (LoRA), WaterSAM incorporates trainable rank decomposition matrices into the Transformer’s layers, specifically enhancing the image encoder. This approach significantly reduces the number of trainable parameters to 6.7% of SAM’s parameters, lowering computational costs. We validated WaterSAM on three underwater image datasets: COD10K, SUIM, and UIIS. Results demonstrate that WaterSAM significantly outperforms pre-trained SAM in underwater segmentation tasks, contributing to advancements in marine biology, underwater archaeology, and environmental monitoring.\",\"PeriodicalId\":16168,\"journal\":{\"name\":\"Journal of Marine Science and Engineering\",\"volume\":\"-1 1\",\"pages\":\"\"},\"PeriodicalIF\":2.7000,\"publicationDate\":\"2024-09-11\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Marine Science and Engineering\",\"FirstCategoryId\":\"89\",\"ListUrlMain\":\"https://doi.org/10.3390/jmse12091616\",\"RegionNum\":3,\"RegionCategory\":\"地球科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ENGINEERING, MARINE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Marine Science and Engineering","FirstCategoryId":"89","ListUrlMain":"https://doi.org/10.3390/jmse12091616","RegionNum":3,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, MARINE","Score":null,"Total":0}
引用次数: 0

摘要

物体分割是图像分割的一种重要类型,主要用于检测和划分图像中的单个物体,对于机器人视觉和增强现实等应用至关重要。尽管深度学习在改进物体分割方面取得了进步,但由于水下特有的复杂性,如湍流扩散、光吸收、噪声、低对比度、光照不均和复杂背景等,水下物体分割仍然具有挑战性。水下数据集的稀缺使这些挑战变得更加复杂。分段任意模型(SAM)已显示出解决这些问题的潜力,但其针对水下环境的改良版 AquaSAM 需要对所有参数进行微调,需要更多的标注数据和高昂的计算成本。在本文中,我们提出了一种用于水下物体分割的适配模型--WaterSAM。受低秩自适应性(Low-Rank Adaptation,LoRA)的启发,WaterSAM 将可训练的秩分解矩阵纳入变换器层,特别增强了图像编码器。这种方法大大减少了可训练参数的数量,仅为 SAM 参数的 6.7%,从而降低了计算成本。我们在三个水下图像数据集上验证了 WaterSAM:COD10K、SUIM 和 UIIS。结果表明,WaterSAM 在水下分割任务中的表现明显优于预训练的 SAM,为海洋生物学、水下考古学和环境监测领域的进步做出了贡献。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
WaterSAM: Adapting SAM for Underwater Object Segmentation
Object segmentation, a key type of image segmentation, focuses on detecting and delineating individual objects within an image, essential for applications like robotic vision and augmented reality. Despite advancements in deep learning improving object segmentation, underwater object segmentation remains challenging due to unique underwater complexities such as turbulence diffusion, light absorption, noise, low contrast, uneven illumination, and intricate backgrounds. The scarcity of underwater datasets further complicates these challenges. The Segment Anything Model (SAM) has shown potential in addressing these issues, but its adaptation for underwater environments, AquaSAM, requires fine-tuning all parameters, demanding more labeled data and high computational costs. In this paper, we propose WaterSAM, an adapted model for underwater object segmentation. Inspired by Low-Rank Adaptation (LoRA), WaterSAM incorporates trainable rank decomposition matrices into the Transformer’s layers, specifically enhancing the image encoder. This approach significantly reduces the number of trainable parameters to 6.7% of SAM’s parameters, lowering computational costs. We validated WaterSAM on three underwater image datasets: COD10K, SUIM, and UIIS. Results demonstrate that WaterSAM significantly outperforms pre-trained SAM in underwater segmentation tasks, contributing to advancements in marine biology, underwater archaeology, and environmental monitoring.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Journal of Marine Science and Engineering
Journal of Marine Science and Engineering Engineering-Ocean Engineering
CiteScore
4.40
自引率
20.70%
发文量
1640
审稿时长
18.09 days
期刊介绍: Journal of Marine Science and Engineering (JMSE; ISSN 2077-1312) is an international, peer-reviewed open access journal which provides an advanced forum for studies related to marine science and engineering. It publishes reviews, research papers and communications. Our aim is to encourage scientists to publish their experimental and theoretical results in as much detail as possible. There is no restriction on the length of the papers. The full experimental details must be provided so that the results can be reproduced. Electronic files and software regarding the full details of the calculation or experimental procedure, if unable to be published in a normal way, can be deposited as supplementary electronic material.
期刊最新文献
Estimation of Source Range and Location Using Ship-Radiated Noise Measured by Two Vertical Line Arrays with a Feed-Forward Neural Network Uncertainty of Wave Spectral Shape and Parameters Associated with the Spectral Estimation Dynamic Response Analysis and Liquefaction Potential Evaluation of Riverbed Induced by Tidal Bore Thermodynamic Analysis of a Marine Diesel Engine Waste Heat-Assisted Cogeneration Power Plant Modified with Regeneration Onboard a Ship Performance of a Cable-Driven Robot Used for Cyber–Physical Testing of Floating Wind Turbines
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1