BiFormer Attention-Guided Multiscale Fusion Mask2former Networks for Fish Abnormal Behavior Recognition and Segmentation

IF 1.9 4区 农林科学 Q2 FISHERIES Aquaculture Research Pub Date : 2024-11-26 DOI:10.1155/are/8892810
Jihang Liu, Zeyuan Hu, Yixi Zhang, Yinjia Li, Jinsong Yang, Hong Yu
{"title":"BiFormer Attention-Guided Multiscale Fusion Mask2former Networks for Fish Abnormal Behavior Recognition and Segmentation","authors":"Jihang Liu,&nbsp;Zeyuan Hu,&nbsp;Yixi Zhang,&nbsp;Yinjia Li,&nbsp;Jinsong Yang,&nbsp;Hong Yu","doi":"10.1155/are/8892810","DOIUrl":null,"url":null,"abstract":"<div>\n <p>To address the issues of accurately identifying and tracking individual fish abnormal behaviors and poor adaptability in the aquaculture field, this paper proposes a Mask2former model combined with a bidirectional routing attention mechanism (BiFormer) and a multiscale dilated attention (MSDA) module for fish abnormal behavior recognition and segmentation. To compensate for the lack of publicly available datasets on fish abnormal behavior, we created the “FISH_segmentation_2023” abnormal behavior dataset, which includes four types of fish behaviors. First, by introducing the BiFormer attention mechanism, the model can better capture critical temporal and spatial information in image sequences, significantly enhancing feature representation. Second, after processing the feature maps with the pixel decoder, the MSDA module is introduced to perform multiscale fusion on these features. The fused features are then passed to the transformer decoder, further enhancing the model’s ability to recognize fish abnormal behaviors. Finally, to further improve model performance and address class imbalance issues in the dataset, we designed a composite loss function combining focal loss and dice loss (FD loss). This loss function can balance the influence of easy and difficult-to-classify samples while optimizing segmentation performance, thereby improving the model’s recognition accuracy and mean intersection over union (mIoU) metrics. Experimental results show that the BiFormer multiscale dilated attention FD loss (BMF)-Mask2former model exhibits high performance, achieving average intersection over union (IoU), accuracy, and recall values of 92.33%, 95.63%, and 94.82%, respectively, on the self-built FISH_segmentation_2023 dataset, representing improvements of 6.10%, 4.50%, and 5.09%, respectively, compared to the Mask2former model. The study demonstrates that the proposed model can accurately capture both local and contextual features of fish abnormal behaviors through multiscale fusion methods, resulting in high-quality segmentation outcomes.</p>\n </div>","PeriodicalId":8104,"journal":{"name":"Aquaculture Research","volume":"2024 1","pages":""},"PeriodicalIF":1.9000,"publicationDate":"2024-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1155/are/8892810","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Aquaculture Research","FirstCategoryId":"97","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1155/are/8892810","RegionNum":4,"RegionCategory":"农林科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"FISHERIES","Score":null,"Total":0}
引用次数: 0

Abstract

To address the issues of accurately identifying and tracking individual fish abnormal behaviors and poor adaptability in the aquaculture field, this paper proposes a Mask2former model combined with a bidirectional routing attention mechanism (BiFormer) and a multiscale dilated attention (MSDA) module for fish abnormal behavior recognition and segmentation. To compensate for the lack of publicly available datasets on fish abnormal behavior, we created the “FISH_segmentation_2023” abnormal behavior dataset, which includes four types of fish behaviors. First, by introducing the BiFormer attention mechanism, the model can better capture critical temporal and spatial information in image sequences, significantly enhancing feature representation. Second, after processing the feature maps with the pixel decoder, the MSDA module is introduced to perform multiscale fusion on these features. The fused features are then passed to the transformer decoder, further enhancing the model’s ability to recognize fish abnormal behaviors. Finally, to further improve model performance and address class imbalance issues in the dataset, we designed a composite loss function combining focal loss and dice loss (FD loss). This loss function can balance the influence of easy and difficult-to-classify samples while optimizing segmentation performance, thereby improving the model’s recognition accuracy and mean intersection over union (mIoU) metrics. Experimental results show that the BiFormer multiscale dilated attention FD loss (BMF)-Mask2former model exhibits high performance, achieving average intersection over union (IoU), accuracy, and recall values of 92.33%, 95.63%, and 94.82%, respectively, on the self-built FISH_segmentation_2023 dataset, representing improvements of 6.10%, 4.50%, and 5.09%, respectively, compared to the Mask2former model. The study demonstrates that the proposed model can accurately capture both local and contextual features of fish abnormal behaviors through multiscale fusion methods, resulting in high-quality segmentation outcomes.

Abstract Image

查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
用于鱼类异常行为识别和分割的 BiFormer Attention-Guided Multiscale Fusion Mask2former 网络
针对水产养殖领域鱼类个体异常行为难以准确识别和跟踪、适应性差等问题,本文提出了一种结合双向路由注意力机制(BiFormer)和多尺度扩张注意力(MSDA)模块的 Mask2former 模型,用于鱼类异常行为识别和分割。为了弥补鱼类异常行为公开数据集的不足,我们创建了 "FISH_segmentation_2023 "异常行为数据集,其中包括四种鱼类行为。首先,通过引入 BiFormer 注意机制,该模型可以更好地捕捉图像序列中关键的时间和空间信息,显著增强特征表示能力。其次,在使用像素解码器处理特征图之后,引入 MSDA 模块对这些特征进行多尺度融合。然后将融合后的特征传递给变换解码器,进一步提高模型识别鱼类异常行为的能力。最后,为了进一步提高模型性能并解决数据集中的类不平衡问题,我们设计了一种结合了焦点损失和骰子损失(FD loss)的复合损失函数。该损失函数可以在优化分割性能的同时,平衡易分类样本和难分类样本的影响,从而提高模型的识别准确率和平均交集大于联合度(mIoU)指标。实验结果表明,BiFormer 多尺度稀释注意力 FD 损失(BMF)- Mask2former 模型表现出了很高的性能,在自建的 FISH_segmentation_2023 数据集上,该模型的平均联合交集(IoU)、准确率和召回率分别达到了 92.33%、95.63% 和 94.82%,与 Mask2former 模型相比,分别提高了 6.10%、4.50% 和 5.09%。该研究表明,所提出的模型可以通过多尺度融合方法准确捕捉鱼类异常行为的局部特征和上下文特征,从而获得高质量的分割结果。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
Aquaculture Research
Aquaculture Research 农林科学-渔业
CiteScore
4.60
自引率
5.00%
发文量
464
审稿时长
5.3 months
期刊介绍: International in perspective, Aquaculture Research is published 12 times a year and specifically addresses research and reference needs of all working and studying within the many varied areas of aquaculture. The Journal regularly publishes papers on applied or scientific research relevant to freshwater, brackish, and marine aquaculture. It covers all aquatic organisms, floristic and faunistic, related directly or indirectly to human consumption. The journal also includes review articles, short communications and technical papers. Young scientists are particularly encouraged to submit short communications based on their own research.
期刊最新文献
Use of Chemical–Physical and Natural Ingredients of Food Factors Modulation for Freshwater Aquaculture Systems Fish Wellness: A Systematic Review Genetic Diversity of Oreochromis shiranus (Boulenger 1905) After Six Consecutive Generations of Selective Breeding Compared With Corresponding Wild Populations BiFormer Attention-Guided Multiscale Fusion Mask2former Networks for Fish Abnormal Behavior Recognition and Segmentation The Multifaceted Role of Gut Microbiota in Sea Urchin Digestion: Diversity, Function, Symbiosis, and Carbohydrate Degradation Insight Into Dietary Utilization of African Palm Weevil (Rhynchophorus phoenicis) Oil: Assessing the Effect on Growth, Serum Biochemistry, Liver, and Intestinal Histomorphology of African Catfish (Clarias gariepinus)
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1