{"title":"天体点源图像分类的多模态迁移学习方法","authors":"Bingjun Wang, Shuxin Hong, Zhiyang Yuan, A-Li Luo, Xiao Kong, Zhiqiang Zou","doi":"10.1088/1538-3873/acfbb9","DOIUrl":null,"url":null,"abstract":"Abstract A large fraction of celestial objects exhibit point shapes in CCD images, such as stars and QSOs, which contain less information due to their few pixels. Point source classification based solely on image data may lead to low accuracy. To address this challenge, this paper proposes a Multi-modal Transfer Learning-based classification method for celestial objects with point shape images. Considering that spectral data possess rich features and that there is a correlation between spectral data and image data, the proposed approach fully utilizes the knowledge gained from celestial spectral data and transfers it to the original image-based classification, enhancing the accuracy of classifying stars and QSOs. Initially, a one-dimensional residual network is employed to extract a 128-dimensional spectral feature vector from the original 3700-dimensional spectral data. This spectral feature vector captures important features of the celestial object. The Generative Adversarial Network is then utilized to generate a simulated spectral vector of 128 dimensions, which corresponds to the celestial object image. By generating simulated spectral vectors, data from two modals (spectral and image) for the same celestial object are available, enriching the input features of the model. In the upcoming multimodal classification model, we only require the images of celestial objects along with their corresponding simulated spectral data, and we no longer need real spectral data. With the assistance of spectral data, the proposed method alleviates the above disadvantages of the original image-based classification method. Remarkably, our method has improved the F1-score from 0.93 to 0.9777, while reducing the error rate in classification by 40%. These enhancements significantly increase the classification accuracy of stars and QSOs, providing strong support for the classification of celestial point sources.","PeriodicalId":20820,"journal":{"name":"Publications of the Astronomical Society of the Pacific","volume":"17 1","pages":"0"},"PeriodicalIF":3.3000,"publicationDate":"2023-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"A Multimodal Transfer Learning Method for Classifying Images of Celestial Point Sources\",\"authors\":\"Bingjun Wang, Shuxin Hong, Zhiyang Yuan, A-Li Luo, Xiao Kong, Zhiqiang Zou\",\"doi\":\"10.1088/1538-3873/acfbb9\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Abstract A large fraction of celestial objects exhibit point shapes in CCD images, such as stars and QSOs, which contain less information due to their few pixels. Point source classification based solely on image data may lead to low accuracy. To address this challenge, this paper proposes a Multi-modal Transfer Learning-based classification method for celestial objects with point shape images. Considering that spectral data possess rich features and that there is a correlation between spectral data and image data, the proposed approach fully utilizes the knowledge gained from celestial spectral data and transfers it to the original image-based classification, enhancing the accuracy of classifying stars and QSOs. Initially, a one-dimensional residual network is employed to extract a 128-dimensional spectral feature vector from the original 3700-dimensional spectral data. This spectral feature vector captures important features of the celestial object. The Generative Adversarial Network is then utilized to generate a simulated spectral vector of 128 dimensions, which corresponds to the celestial object image. By generating simulated spectral vectors, data from two modals (spectral and image) for the same celestial object are available, enriching the input features of the model. In the upcoming multimodal classification model, we only require the images of celestial objects along with their corresponding simulated spectral data, and we no longer need real spectral data. With the assistance of spectral data, the proposed method alleviates the above disadvantages of the original image-based classification method. Remarkably, our method has improved the F1-score from 0.93 to 0.9777, while reducing the error rate in classification by 40%. These enhancements significantly increase the classification accuracy of stars and QSOs, providing strong support for the classification of celestial point sources.\",\"PeriodicalId\":20820,\"journal\":{\"name\":\"Publications of the Astronomical Society of the Pacific\",\"volume\":\"17 1\",\"pages\":\"0\"},\"PeriodicalIF\":3.3000,\"publicationDate\":\"2023-10-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Publications of the Astronomical Society of the Pacific\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1088/1538-3873/acfbb9\",\"RegionNum\":3,\"RegionCategory\":\"物理与天体物理\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"ASTRONOMY & ASTROPHYSICS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Publications of the Astronomical Society of the Pacific","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1088/1538-3873/acfbb9","RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ASTRONOMY & ASTROPHYSICS","Score":null,"Total":0}
A Multimodal Transfer Learning Method for Classifying Images of Celestial Point Sources
Abstract A large fraction of celestial objects exhibit point shapes in CCD images, such as stars and QSOs, which contain less information due to their few pixels. Point source classification based solely on image data may lead to low accuracy. To address this challenge, this paper proposes a Multi-modal Transfer Learning-based classification method for celestial objects with point shape images. Considering that spectral data possess rich features and that there is a correlation between spectral data and image data, the proposed approach fully utilizes the knowledge gained from celestial spectral data and transfers it to the original image-based classification, enhancing the accuracy of classifying stars and QSOs. Initially, a one-dimensional residual network is employed to extract a 128-dimensional spectral feature vector from the original 3700-dimensional spectral data. This spectral feature vector captures important features of the celestial object. The Generative Adversarial Network is then utilized to generate a simulated spectral vector of 128 dimensions, which corresponds to the celestial object image. By generating simulated spectral vectors, data from two modals (spectral and image) for the same celestial object are available, enriching the input features of the model. In the upcoming multimodal classification model, we only require the images of celestial objects along with their corresponding simulated spectral data, and we no longer need real spectral data. With the assistance of spectral data, the proposed method alleviates the above disadvantages of the original image-based classification method. Remarkably, our method has improved the F1-score from 0.93 to 0.9777, while reducing the error rate in classification by 40%. These enhancements significantly increase the classification accuracy of stars and QSOs, providing strong support for the classification of celestial point sources.
期刊介绍:
The Publications of the Astronomical Society of the Pacific (PASP), the technical journal of the Astronomical Society of the Pacific (ASP), has been published regularly since 1889, and is an integral part of the ASP''s mission to advance the science of astronomy and disseminate astronomical information. The journal provides an outlet for astronomical results of a scientific nature and serves to keep readers in touch with current astronomical research. It contains refereed research and instrumentation articles, invited and contributed reviews, tutorials, and dissertation summaries.