{"title":"MTPret:利用多任务预训练提高 X 射线图像分析能力","authors":"Weibin Liao;Qingzhong Wang;Xuhong Li;Yi Liu;Zeyu Chen;Siyu Huang;Dejing Dou;Yanwu Xu;Haoyi Xiong","doi":"10.1109/TAI.2024.3400750","DOIUrl":null,"url":null,"abstract":"While deep neural networks (DNNs) have been widely used in various X-ray image analytics tasks such as classification, segmentation, detection, etc., there frequently needs to collect and annotate a huge amount of training data to train a model for every single task. In this work, we proposed a multitask self-supervised pretraining strategy \n<italic>MTPret</i>\n to improve the performance of DNNs in various X-ray analytics tasks. \n<italic>MTPret</i>\n first trains the backbone to learn visual representations from multiple datasets of different tasks through contrastive learning, then \n<italic>MTPret</i>\n leverages a multitask continual learning to learn discriminative features from various downstream tasks. To evaluate the performance of \n<italic>MTPret</i>\n, we collected eleven X-ray image datasets from different body parts, such as heads, chest, lungs, bones, and etc., for various tasks to pretrain backbones, and fine-tuned the networks on seven of the tasks. The evaluation results on top of the seven tasks showed \n<italic>MTPret</i>\n outperformed a large number of baseline methods, including other initialization strategies, pretrained models, and task-specific algorithms in recent studies. In addition, we also performed experiments based on two external tasks, where the datasets of external tasks have not been used in pretraining. The excellent performance of \n<italic>MTPret</i>\n further confirmed the generalizability and superiority of the proposed multitask self-supervised pretraining.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-03-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"MTPret: Improving X-Ray Image Analytics With Multitask Pretraining\",\"authors\":\"Weibin Liao;Qingzhong Wang;Xuhong Li;Yi Liu;Zeyu Chen;Siyu Huang;Dejing Dou;Yanwu Xu;Haoyi Xiong\",\"doi\":\"10.1109/TAI.2024.3400750\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"While deep neural networks (DNNs) have been widely used in various X-ray image analytics tasks such as classification, segmentation, detection, etc., there frequently needs to collect and annotate a huge amount of training data to train a model for every single task. In this work, we proposed a multitask self-supervised pretraining strategy \\n<italic>MTPret</i>\\n to improve the performance of DNNs in various X-ray analytics tasks. \\n<italic>MTPret</i>\\n first trains the backbone to learn visual representations from multiple datasets of different tasks through contrastive learning, then \\n<italic>MTPret</i>\\n leverages a multitask continual learning to learn discriminative features from various downstream tasks. To evaluate the performance of \\n<italic>MTPret</i>\\n, we collected eleven X-ray image datasets from different body parts, such as heads, chest, lungs, bones, and etc., for various tasks to pretrain backbones, and fine-tuned the networks on seven of the tasks. The evaluation results on top of the seven tasks showed \\n<italic>MTPret</i>\\n outperformed a large number of baseline methods, including other initialization strategies, pretrained models, and task-specific algorithms in recent studies. In addition, we also performed experiments based on two external tasks, where the datasets of external tasks have not been used in pretraining. The excellent performance of \\n<italic>MTPret</i>\\n further confirmed the generalizability and superiority of the proposed multitask self-supervised pretraining.\",\"PeriodicalId\":73305,\"journal\":{\"name\":\"IEEE transactions on artificial intelligence\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-03-15\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE transactions on artificial intelligence\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/10531186/\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE transactions on artificial intelligence","FirstCategoryId":"1085","ListUrlMain":"https://ieeexplore.ieee.org/document/10531186/","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0
摘要
虽然深度神经网络(DNN)已被广泛应用于分类、分割、检测等各种 X 射线图像分析任务中,但要为每个任务训练一个模型,往往需要收集和注释大量训练数据。在这项工作中,我们提出了一种多任务自监督预训练策略 MTPret,以提高 DNNs 在各种 X 射线分析任务中的性能。MTPret 首先通过对比学习训练骨干网络从不同任务的多个数据集中学习视觉表征,然后利用多任务持续学习从各种下游任务中学习判别特征。为了评估 MTPret 的性能,我们收集了 11 个不同身体部位(如头部、胸部、肺部、骨骼等)的 X 射线图像数据集,针对不同任务对骨干进行预训练,并在其中 7 个任务上对网络进行了微调。对七项任务的评估结果显示,MTPret优于大量基线方法,包括其他初始化策略、预训练模型和近期研究中的特定任务算法。此外,我们还基于两个外部任务进行了实验,其中外部任务的数据集未用于预训练。MTPret 的出色表现进一步证实了所提出的多任务自监督预训练的普适性和优越性。
MTPret: Improving X-Ray Image Analytics With Multitask Pretraining
While deep neural networks (DNNs) have been widely used in various X-ray image analytics tasks such as classification, segmentation, detection, etc., there frequently needs to collect and annotate a huge amount of training data to train a model for every single task. In this work, we proposed a multitask self-supervised pretraining strategy
MTPret
to improve the performance of DNNs in various X-ray analytics tasks.
MTPret
first trains the backbone to learn visual representations from multiple datasets of different tasks through contrastive learning, then
MTPret
leverages a multitask continual learning to learn discriminative features from various downstream tasks. To evaluate the performance of
MTPret
, we collected eleven X-ray image datasets from different body parts, such as heads, chest, lungs, bones, and etc., for various tasks to pretrain backbones, and fine-tuned the networks on seven of the tasks. The evaluation results on top of the seven tasks showed
MTPret
outperformed a large number of baseline methods, including other initialization strategies, pretrained models, and task-specific algorithms in recent studies. In addition, we also performed experiments based on two external tasks, where the datasets of external tasks have not been used in pretraining. The excellent performance of
MTPret
further confirmed the generalizability and superiority of the proposed multitask self-supervised pretraining.