Suzhen Lin , Songwang Tian , Xiaofei Lu , Dawei Li , Yanbo Wang , Dong Yu
{"title":"基于任务相互依赖的多波段图像同步融合模型","authors":"Suzhen Lin , Songwang Tian , Xiaofei Lu , Dawei Li , Yanbo Wang , Dong Yu","doi":"10.1016/j.ijleo.2024.171937","DOIUrl":null,"url":null,"abstract":"<div><p>Synchronous multi-band image fusion is a challenging, yet urgent task in the development of high-precision detection systems. This study proposes a novel method for synchronous fusion modeling of multi-band images based on task-interdependency. In the proposed method, the task of image fusion is divided into two mutually exclusive sub-tasks that produce bright thermal targets and obtain precise textural details. First, two generators with different network structures and several discriminators produce a preliminary fused image. Second, an image fusion strategy is defined using a model- and data-driven theory to obtain fused images. Then, each discriminator classifies the fused image and source images of each band to force the generators to produce the desired results. A novel loss function is constructed to enhance the fused effect by selecting the most significant gradient loss and loss of brightness. Finally, the network is trained based on a multi-generative adversarial framework.The trained generators can be used individually or jointly as a model for fusing multiple images. We verified our method with several datasets and determined that it outperforms other current methods.</p></div>","PeriodicalId":19513,"journal":{"name":"Optik","volume":null,"pages":null},"PeriodicalIF":3.1000,"publicationDate":"2024-06-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Multi-band image synchronous fusion model based on task-interdependency\",\"authors\":\"Suzhen Lin , Songwang Tian , Xiaofei Lu , Dawei Li , Yanbo Wang , Dong Yu\",\"doi\":\"10.1016/j.ijleo.2024.171937\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><p>Synchronous multi-band image fusion is a challenging, yet urgent task in the development of high-precision detection systems. This study proposes a novel method for synchronous fusion modeling of multi-band images based on task-interdependency. In the proposed method, the task of image fusion is divided into two mutually exclusive sub-tasks that produce bright thermal targets and obtain precise textural details. First, two generators with different network structures and several discriminators produce a preliminary fused image. Second, an image fusion strategy is defined using a model- and data-driven theory to obtain fused images. Then, each discriminator classifies the fused image and source images of each band to force the generators to produce the desired results. A novel loss function is constructed to enhance the fused effect by selecting the most significant gradient loss and loss of brightness. Finally, the network is trained based on a multi-generative adversarial framework.The trained generators can be used individually or jointly as a model for fusing multiple images. We verified our method with several datasets and determined that it outperforms other current methods.</p></div>\",\"PeriodicalId\":19513,\"journal\":{\"name\":\"Optik\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":3.1000,\"publicationDate\":\"2024-06-28\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Optik\",\"FirstCategoryId\":\"101\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S003040262400336X\",\"RegionNum\":3,\"RegionCategory\":\"物理与天体物理\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"Engineering\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Optik","FirstCategoryId":"101","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S003040262400336X","RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"Engineering","Score":null,"Total":0}
Multi-band image synchronous fusion model based on task-interdependency
Synchronous multi-band image fusion is a challenging, yet urgent task in the development of high-precision detection systems. This study proposes a novel method for synchronous fusion modeling of multi-band images based on task-interdependency. In the proposed method, the task of image fusion is divided into two mutually exclusive sub-tasks that produce bright thermal targets and obtain precise textural details. First, two generators with different network structures and several discriminators produce a preliminary fused image. Second, an image fusion strategy is defined using a model- and data-driven theory to obtain fused images. Then, each discriminator classifies the fused image and source images of each band to force the generators to produce the desired results. A novel loss function is constructed to enhance the fused effect by selecting the most significant gradient loss and loss of brightness. Finally, the network is trained based on a multi-generative adversarial framework.The trained generators can be used individually or jointly as a model for fusing multiple images. We verified our method with several datasets and determined that it outperforms other current methods.
期刊介绍:
Optik publishes articles on all subjects related to light and electron optics and offers a survey on the state of research and technical development within the following fields:
Optics:
-Optics design, geometrical and beam optics, wave optics-
Optical and micro-optical components, diffractive optics, devices and systems-
Photoelectric and optoelectronic devices-
Optical properties of materials, nonlinear optics, wave propagation and transmission in homogeneous and inhomogeneous materials-
Information optics, image formation and processing, holographic techniques, microscopes and spectrometer techniques, and image analysis-
Optical testing and measuring techniques-
Optical communication and computing-
Physiological optics-
As well as other related topics.