利用自知识精馏提高小样本图像分类的泛化性能

IF 1.2 4区 计算机科学 Q4 AUTOMATION & CONTROL SYSTEMS Studies in Informatics and Control Pub Date : 2022-06-30 DOI:10.24846/v31i2y202207
Liang Li, Weidong Jin, Yingkun Huang, Junxiao Ren
{"title":"利用自知识精馏提高小样本图像分类的泛化性能","authors":"Liang Li, Weidong Jin, Yingkun Huang, Junxiao Ren","doi":"10.24846/v31i2y202207","DOIUrl":null,"url":null,"abstract":": Though deep learning has succeeded in various fields, its performance on tasks without a large-scale dataset is always unsatisfactory. The meta-learning based few-shot learning has been used to address the limited data situation. Because of its fast adaptation to the new concepts, meta-learning fully utilizes the prior transferrable knowledge to recognize the unseen instances. The general belief is that meta-learning leverages a large quantity of few-shot tasks sampled from the base dataset to quickly adapt the learner to an unseen task. In this paper, the teacher model is distilled to transfer the features using the same architecture. Following the standard-setting in few-shot learning, the proposed model was trained from scratch and the distribution was transferred to a better generalization. Feature similarity matching was proposed to compensate for the inner feature similarities. Besides, the prediction from the teacher model was further corrected in the self-knowledge distillation period. The proposed approach was evaluated on several commonly used benchmarks in few-shot learning and performed best among all prior works.","PeriodicalId":49466,"journal":{"name":"Studies in Informatics and Control","volume":" ","pages":""},"PeriodicalIF":1.2000,"publicationDate":"2022-06-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Enhancing the Generalization Performance of Few-Shot Image Classification with Self-Knowledge Distillation\",\"authors\":\"Liang Li, Weidong Jin, Yingkun Huang, Junxiao Ren\",\"doi\":\"10.24846/v31i2y202207\",\"DOIUrl\":null,\"url\":null,\"abstract\":\": Though deep learning has succeeded in various fields, its performance on tasks without a large-scale dataset is always unsatisfactory. The meta-learning based few-shot learning has been used to address the limited data situation. Because of its fast adaptation to the new concepts, meta-learning fully utilizes the prior transferrable knowledge to recognize the unseen instances. The general belief is that meta-learning leverages a large quantity of few-shot tasks sampled from the base dataset to quickly adapt the learner to an unseen task. In this paper, the teacher model is distilled to transfer the features using the same architecture. Following the standard-setting in few-shot learning, the proposed model was trained from scratch and the distribution was transferred to a better generalization. Feature similarity matching was proposed to compensate for the inner feature similarities. Besides, the prediction from the teacher model was further corrected in the self-knowledge distillation period. The proposed approach was evaluated on several commonly used benchmarks in few-shot learning and performed best among all prior works.\",\"PeriodicalId\":49466,\"journal\":{\"name\":\"Studies in Informatics and Control\",\"volume\":\" \",\"pages\":\"\"},\"PeriodicalIF\":1.2000,\"publicationDate\":\"2022-06-30\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Studies in Informatics and Control\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://doi.org/10.24846/v31i2y202207\",\"RegionNum\":4,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q4\",\"JCRName\":\"AUTOMATION & CONTROL SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Studies in Informatics and Control","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.24846/v31i2y202207","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q4","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}
引用次数: 0

摘要

:尽管深度学习在各个领域都取得了成功,但在没有大规模数据集的情况下,它在任务中的表现总是令人不满意。基于元学习的少镜头学习已被用于解决数据有限的情况。由于元学习对新概念的快速适应,它充分利用了先前可转移的知识来识别看不见的实例。人们普遍认为,元学习利用了从基本数据集中采样的大量少镜头任务,使学习者快速适应看不见的任务。在本文中,教师模型被提炼出来,以使用相同的架构来传递特征。根据少镜头学习中的标准设置,从零开始训练所提出的模型,并将分布转换为更好的泛化。提出了一种特征相似度匹配方法来补偿内部特征相似度。此外,教师模型的预测在自学阶段得到了进一步的修正。所提出的方法在少镜头学习中的几个常用基准上进行了评估,在所有先前的工作中表现最好。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
Enhancing the Generalization Performance of Few-Shot Image Classification with Self-Knowledge Distillation
: Though deep learning has succeeded in various fields, its performance on tasks without a large-scale dataset is always unsatisfactory. The meta-learning based few-shot learning has been used to address the limited data situation. Because of its fast adaptation to the new concepts, meta-learning fully utilizes the prior transferrable knowledge to recognize the unseen instances. The general belief is that meta-learning leverages a large quantity of few-shot tasks sampled from the base dataset to quickly adapt the learner to an unseen task. In this paper, the teacher model is distilled to transfer the features using the same architecture. Following the standard-setting in few-shot learning, the proposed model was trained from scratch and the distribution was transferred to a better generalization. Feature similarity matching was proposed to compensate for the inner feature similarities. Besides, the prediction from the teacher model was further corrected in the self-knowledge distillation period. The proposed approach was evaluated on several commonly used benchmarks in few-shot learning and performed best among all prior works.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Studies in Informatics and Control
Studies in Informatics and Control AUTOMATION & CONTROL SYSTEMS-OPERATIONS RESEARCH & MANAGEMENT SCIENCE
CiteScore
2.70
自引率
25.00%
发文量
34
审稿时长
>12 weeks
期刊介绍: Studies in Informatics and Control journal provides important perspectives on topics relevant to Information Technology, with an emphasis on useful applications in the most important areas of IT. This journal is aimed at advanced practitioners and researchers in the field of IT and welcomes original contributions from scholars and professionals worldwide. SIC is published both in print and online by the National Institute for R&D in Informatics, ICI Bucharest. Abstracts, full text and graphics of all articles in the online version of SIC are identical to the print version of the Journal.
期刊最新文献
Event-Triggered Piecewise Continuous Tracking Control of Networked Control Systems using Linear Perturbed System Models with Time Delays Development of Hybrid Model based on Artificial Intelligence for Maximizing Solar Energy Yield Optimal Second Order Sliding Control for the Robust Tracking of a 2-Degree-of-Freedom Helicopter System based on Metaheuristics and Artificial Neural Networks Modeling of a Hybrid Controller for Electric Vehicle Battery Charging Using Photovoltaic Panels Duty Cycles Mathematical Analysis and Empirical Thrust-Force Performance Curves of a Brushless Electric Motor
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1