DDOWOD: DiffusionDet for open-world object detection

IF 3.9 3区 计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE Pattern Recognition Letters Pub Date : 2024-10-01 DOI:10.1016/j.patrec.2024.10.002
Jiaqi Fan , Enming Zhang , Ying Wei , Yuefeng Wang , Jiakun Xia , Junwei Liu , Xinghong Liu , Shuailei Ma
{"title":"DDOWOD: DiffusionDet for open-world object detection","authors":"Jiaqi Fan ,&nbsp;Enming Zhang ,&nbsp;Ying Wei ,&nbsp;Yuefeng Wang ,&nbsp;Jiakun Xia ,&nbsp;Junwei Liu ,&nbsp;Xinghong Liu ,&nbsp;Shuailei Ma","doi":"10.1016/j.patrec.2024.10.002","DOIUrl":null,"url":null,"abstract":"<div><div>Open-world object detection (OWOD) poses a significant challenge in computer vision, requiring models to detect unknown objects and incrementally learn new categories. To explore this field, we propose the DDOWOD based on the DiffusionDet. It is more likely to cover unknown objects hidden in the background and can reduce the model’s bias towards known class objects during training due to its ability to randomly generate boxes and reconstruct the characteristics of the GT from them. Also, to improve the insufficient quality of pseudo-labels which leads to reduced accuracy in recognizing unknown classes, we use the Segment Anything Model (SAM) as the teacher model in distillation learning to endow DDOWOD with rich visual knowledge. Surprisingly, compared to other existing models, our DDOWOD is more suitable for using SAM as the teacher. Furthermore, we proposed the Stepwise distillation (SD) which is a new incremental learning method specialized for our DDOWOD to avoid catastrophic forgetting during the training. Our approach utilizes all previously trained models from past tasks rather than solely relying on the last one. DDOWOD has achieved excellent performance. U-Recall is 53.2, 51.5, 50.7 in OWOD split and U-AP is 21.9 in IntensiveSet.</div></div>","PeriodicalId":54638,"journal":{"name":"Pattern Recognition Letters","volume":"186 ","pages":"Pages 170-177"},"PeriodicalIF":3.9000,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Pattern Recognition Letters","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0167865524002903","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0

Abstract

Open-world object detection (OWOD) poses a significant challenge in computer vision, requiring models to detect unknown objects and incrementally learn new categories. To explore this field, we propose the DDOWOD based on the DiffusionDet. It is more likely to cover unknown objects hidden in the background and can reduce the model’s bias towards known class objects during training due to its ability to randomly generate boxes and reconstruct the characteristics of the GT from them. Also, to improve the insufficient quality of pseudo-labels which leads to reduced accuracy in recognizing unknown classes, we use the Segment Anything Model (SAM) as the teacher model in distillation learning to endow DDOWOD with rich visual knowledge. Surprisingly, compared to other existing models, our DDOWOD is more suitable for using SAM as the teacher. Furthermore, we proposed the Stepwise distillation (SD) which is a new incremental learning method specialized for our DDOWOD to avoid catastrophic forgetting during the training. Our approach utilizes all previously trained models from past tasks rather than solely relying on the last one. DDOWOD has achieved excellent performance. U-Recall is 53.2, 51.5, 50.7 in OWOD split and U-AP is 21.9 in IntensiveSet.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
DDOWOD:用于开放世界物体检测的 DiffusionDet
开放世界物体检测(OWOD)是计算机视觉领域的一项重大挑战,需要模型检测未知物体并逐步学习新的类别。为了探索这一领域,我们提出了基于 DiffusionDet 的 DDOWOD,它更有可能覆盖隐藏在背景中的未知物体,并能随机生成方框并从中重建 GT 的特征,从而减少模型在训练过程中对已知类别物体的偏差。同时,为了改善伪标签质量不足导致识别未知类别准确率降低的问题,我们在蒸馏学习中使用了 "任意分段模型"(Segment Anything Model,SAM)作为教师模型,为 DDOWOD 赋予丰富的视觉知识。令人惊讶的是,与其他现有模型相比,我们的 DDOWOD 更适合使用 SAM 作为教师模型。此外,我们还提出了逐步蒸馏法(SD),这是一种新的增量学习方法,专门用于我们的 DDOWOD,以避免训练过程中的灾难性遗忘。我们的方法利用了以往任务中所有训练过的模型,而不是仅仅依赖于最后一个模型。DDOWOD 取得了优异的性能。在 OWOD 分案中,U-Recall 为 53.2,51.5,50.7;在 IntensiveSet 中,U-AP 为 21.9。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
Pattern Recognition Letters
Pattern Recognition Letters 工程技术-计算机:人工智能
CiteScore
12.40
自引率
5.90%
发文量
287
审稿时长
9.1 months
期刊介绍: Pattern Recognition Letters aims at rapid publication of concise articles of a broad interest in pattern recognition. Subject areas include all the current fields of interest represented by the Technical Committees of the International Association of Pattern Recognition, and other developing themes involving learning and recognition.
期刊最新文献
Prototypical class-wise test-time adaptation Improving ViT interpretability with patch-level mask prediction Sparse-attention augmented domain adaptation for unsupervised person re-identification GANzzle++: Generative approaches for jigsaw puzzle solving as local to global assignment in latent spatial representations Neuromorphic face analysis: A survey
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1