Yating Liu, Lulu Fan, Lei Hu, Junqiang Lu, Yan Lu, Zelin Xu, Jiazheng Zhu, Haochen Wang, Xu Kong
{"title":"利用主动学习和半监督学习对真实和虚假瞬态进行分类","authors":"Yating Liu, Lulu Fan, Lei Hu, Junqiang Lu, Yan Lu, Zelin Xu, Jiazheng Zhu, Haochen Wang, Xu Kong","doi":"10.1051/0004-6361/202348581","DOIUrl":null,"url":null,"abstract":"<i>Context<i/>. The mounting data stream of large time-domain surveys renders the visual inspections of a huge set of transient candidates impractical. Techniques based on deep learning-based are popular solutions for minimizing human intervention in the time domain community. The classification of real and bogus transients is a fundamental component in real-time data processing systems and is critical to enabling rapid follow-up observations. Most existing methods (supervised learning) require sufficiently large training samples with corresponding labels, which involve costly human labeling and are challenging in the early stages of a time-domain survey. One method that can make use of training samples with access to only a limited amount of labels is highly desirable for future large time-domain surveys. These include the forthcoming 2.5-meter Wide-Field Survey Telescope (WFST) six-year survey and the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST).<i>Aims<i/>. Deep-learning-based methods have been favored in astrophysics owing to their adaptability and remarkable performance. They have been applied to the task of the classification of real and bogus transients. Unlike most existing approaches, which necessitate massive and expensive annotated data, we aim to leverage training samples with only 1000 labels and discover real sources that vary in brightness over time in the early stages of the WFST six-year survey.<i>Methods<i/>. We present a novel deep learning method that combines active learning and semi-supervised learning to construct a competitive real-bogus classifier. Our method incorporates an active learning stage, where we actively select the most informative or uncertain samples for annotation. This stage aims to achieve higher model performance by leveraging fewer labeled samples, thus reducing annotation costs and improving the overall learning process efficiency. Furthermore, our approach involves a semi-supervised learning stage that exploits the unlabeled data to enhance the model’s performance and achieve superior results, compared to using only the limited labeled data.<i>Results<i/>. Our proposed methodology capitalizes on the potential of active learning and semi-supervised learning. To demonstrate the efficacy of our approach, we constructed three newly compiled datasets from the Zwicky Transient Facility, achieving average accuracies of 98.8, 98.8, and 98.6% across these three datasets. It is important to note that our newly compiled datasets only work in terms of testing our deep learning methodology and there may be a potential bias between our datasets and the complete data stream. Therefore, the observed performance on these datasets cannot be assumed to directly translate to the general alert stream for general transient detection in actual scenarios. The algorithm will be integrated into the WFST pipeline, enabling an efficient and effective classification of transients in the early period of a time-domain survey.","PeriodicalId":8571,"journal":{"name":"Astronomy & Astrophysics","volume":"13 1","pages":""},"PeriodicalIF":5.4000,"publicationDate":"2025-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Classification of real and bogus transients using active learning and semi-supervised learning\",\"authors\":\"Yating Liu, Lulu Fan, Lei Hu, Junqiang Lu, Yan Lu, Zelin Xu, Jiazheng Zhu, Haochen Wang, Xu Kong\",\"doi\":\"10.1051/0004-6361/202348581\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<i>Context<i/>. The mounting data stream of large time-domain surveys renders the visual inspections of a huge set of transient candidates impractical. Techniques based on deep learning-based are popular solutions for minimizing human intervention in the time domain community. The classification of real and bogus transients is a fundamental component in real-time data processing systems and is critical to enabling rapid follow-up observations. Most existing methods (supervised learning) require sufficiently large training samples with corresponding labels, which involve costly human labeling and are challenging in the early stages of a time-domain survey. One method that can make use of training samples with access to only a limited amount of labels is highly desirable for future large time-domain surveys. These include the forthcoming 2.5-meter Wide-Field Survey Telescope (WFST) six-year survey and the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST).<i>Aims<i/>. Deep-learning-based methods have been favored in astrophysics owing to their adaptability and remarkable performance. They have been applied to the task of the classification of real and bogus transients. Unlike most existing approaches, which necessitate massive and expensive annotated data, we aim to leverage training samples with only 1000 labels and discover real sources that vary in brightness over time in the early stages of the WFST six-year survey.<i>Methods<i/>. We present a novel deep learning method that combines active learning and semi-supervised learning to construct a competitive real-bogus classifier. Our method incorporates an active learning stage, where we actively select the most informative or uncertain samples for annotation. This stage aims to achieve higher model performance by leveraging fewer labeled samples, thus reducing annotation costs and improving the overall learning process efficiency. Furthermore, our approach involves a semi-supervised learning stage that exploits the unlabeled data to enhance the model’s performance and achieve superior results, compared to using only the limited labeled data.<i>Results<i/>. Our proposed methodology capitalizes on the potential of active learning and semi-supervised learning. To demonstrate the efficacy of our approach, we constructed three newly compiled datasets from the Zwicky Transient Facility, achieving average accuracies of 98.8, 98.8, and 98.6% across these three datasets. It is important to note that our newly compiled datasets only work in terms of testing our deep learning methodology and there may be a potential bias between our datasets and the complete data stream. Therefore, the observed performance on these datasets cannot be assumed to directly translate to the general alert stream for general transient detection in actual scenarios. The algorithm will be integrated into the WFST pipeline, enabling an efficient and effective classification of transients in the early period of a time-domain survey.\",\"PeriodicalId\":8571,\"journal\":{\"name\":\"Astronomy & Astrophysics\",\"volume\":\"13 1\",\"pages\":\"\"},\"PeriodicalIF\":5.4000,\"publicationDate\":\"2025-01-09\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Astronomy & Astrophysics\",\"FirstCategoryId\":\"101\",\"ListUrlMain\":\"https://doi.org/10.1051/0004-6361/202348581\",\"RegionNum\":2,\"RegionCategory\":\"物理与天体物理\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ASTRONOMY & ASTROPHYSICS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Astronomy & Astrophysics","FirstCategoryId":"101","ListUrlMain":"https://doi.org/10.1051/0004-6361/202348581","RegionNum":2,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ASTRONOMY & ASTROPHYSICS","Score":null,"Total":0}
引用次数: 0
摘要
上下文。大型时域调查的数据流不断增加,使得对大量瞬态候选者的目视检查变得不切实际。基于深度学习的技术是在时域社区中减少人为干预的流行解决方案。真实瞬变和虚假瞬变的分类是实时数据处理系统的基本组成部分,对于实现快速后续观测至关重要。大多数现有的方法(监督学习)需要足够大的具有相应标签的训练样本,这涉及到昂贵的人工标记,并且在时域调查的早期阶段具有挑战性。有一种方法可以利用只能访问有限数量标签的训练样本,这对于未来的大型时域调查是非常可取的。其中包括即将到来的2.5米宽视场巡天望远镜(WFST)六年巡天计划和Vera C. Rubin天文台的时空遗产巡天计划(LSST)。基于深度学习的方法因其适应性和卓越的性能而受到天体物理学领域的青睐。它们已被应用于真实瞬变和虚假瞬变的分类任务。与大多数现有方法不同,这些方法需要大量且昂贵的注释数据,我们的目标是利用只有1000个标签的训练样本,并在WFST六年调查的早期阶段发现亮度随时间变化的真实来源。提出了一种新的深度学习方法,将主动学习和半监督学习相结合,构建了一个竞争真伪分类器。我们的方法包含了一个主动学习阶段,在这个阶段,我们主动选择最有信息或最不确定的样本进行注释。该阶段旨在利用更少的标记样本来获得更高的模型性能,从而降低标注成本,提高整体学习过程效率。此外,我们的方法涉及半监督学习阶段,与仅使用有限的标记数据相比,该阶段利用未标记数据来增强模型的性能并获得更好的结果。我们提出的方法利用了主动学习和半监督学习的潜力。为了证明我们方法的有效性,我们从Zwicky瞬态设施构建了三个新编译的数据集,在这三个数据集中实现了98.8%,98.8%和98.6%的平均精度。值得注意的是,我们新编译的数据集仅用于测试我们的深度学习方法,并且我们的数据集和完整的数据流之间可能存在潜在的偏差。因此,不能假设在这些数据集上观察到的性能直接转化为实际场景中用于一般瞬态检测的一般警报流。该算法将集成到WFST管道中,在时域调查的早期阶段实现高效和有效的瞬态分类。
Classification of real and bogus transients using active learning and semi-supervised learning
Context. The mounting data stream of large time-domain surveys renders the visual inspections of a huge set of transient candidates impractical. Techniques based on deep learning-based are popular solutions for minimizing human intervention in the time domain community. The classification of real and bogus transients is a fundamental component in real-time data processing systems and is critical to enabling rapid follow-up observations. Most existing methods (supervised learning) require sufficiently large training samples with corresponding labels, which involve costly human labeling and are challenging in the early stages of a time-domain survey. One method that can make use of training samples with access to only a limited amount of labels is highly desirable for future large time-domain surveys. These include the forthcoming 2.5-meter Wide-Field Survey Telescope (WFST) six-year survey and the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST).Aims. Deep-learning-based methods have been favored in astrophysics owing to their adaptability and remarkable performance. They have been applied to the task of the classification of real and bogus transients. Unlike most existing approaches, which necessitate massive and expensive annotated data, we aim to leverage training samples with only 1000 labels and discover real sources that vary in brightness over time in the early stages of the WFST six-year survey.Methods. We present a novel deep learning method that combines active learning and semi-supervised learning to construct a competitive real-bogus classifier. Our method incorporates an active learning stage, where we actively select the most informative or uncertain samples for annotation. This stage aims to achieve higher model performance by leveraging fewer labeled samples, thus reducing annotation costs and improving the overall learning process efficiency. Furthermore, our approach involves a semi-supervised learning stage that exploits the unlabeled data to enhance the model’s performance and achieve superior results, compared to using only the limited labeled data.Results. Our proposed methodology capitalizes on the potential of active learning and semi-supervised learning. To demonstrate the efficacy of our approach, we constructed three newly compiled datasets from the Zwicky Transient Facility, achieving average accuracies of 98.8, 98.8, and 98.6% across these three datasets. It is important to note that our newly compiled datasets only work in terms of testing our deep learning methodology and there may be a potential bias between our datasets and the complete data stream. Therefore, the observed performance on these datasets cannot be assumed to directly translate to the general alert stream for general transient detection in actual scenarios. The algorithm will be integrated into the WFST pipeline, enabling an efficient and effective classification of transients in the early period of a time-domain survey.
期刊介绍:
Astronomy & Astrophysics is an international Journal that publishes papers on all aspects of astronomy and astrophysics (theoretical, observational, and instrumental) independently of the techniques used to obtain the results.