Information Fusion最新文献_第4页

Applications of knowledge distillation in remote sensing: A survey 遥感中的知识提炼应用：调查

IF 14.7 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2024-10-19 DOI: 10.1016/j.inffus.2024.102742

Yassine Himeur , Nour Aburaed , Omar Elharrouss , Iraklis Varlamis , Shadi Atalla , Wathiq Mansoor , Hussain Al-Ahmad

With the ever-growing complexity of models in the field of remote sensing (RS), there is an increasing demand for solutions that balance model accuracy with computational efficiency. Knowledge distillation (KD) has emerged as a powerful tool to meet this need, enabling the transfer of knowledge from large, complex models to smaller, more efficient ones without significant loss in performance. This review article provides an extensive examination of KD and its innovative applications in RS. KD, a technique developed to transfer knowledge from a complex, often cumbersome model (teacher) to a more compact and efficient model (student), has seen significant evolution and application across various domains. Initially, we introduce the fundamental concepts and historical progression of KD methods. The advantages of employing KD are highlighted, particularly in terms of model compression, enhanced computational efficiency, and improved performance, which are pivotal for practical deployments in RS scenarios. The article provides a comprehensive taxonomy of KD techniques, where each category is critically analyzed to demonstrate the breadth and depth of the alternative options, and illustrates specific case studies that showcase the practical implementation of KD methods in RS tasks, such as instance segmentation and object detection. Further, the review discusses the challenges and limitations of KD in RS, including practical constraints and prospective future directions, providing a comprehensive overview for researchers and practitioners in the field of RS. Through this organization, the paper not only elucidates the current state of research in KD but also sets the stage for future research opportunities, thereby contributing significantly to both academic research and real-world applications.

随着遥感（RS）领域模型的复杂性不断增加，对兼顾模型准确性和计算效率的解决方案的需求也越来越大。知识蒸馏（KD）已成为满足这一需求的有力工具，它能将知识从大型、复杂的模型转移到更小、更高效的模型，而不会明显降低性能。这篇综述文章对 KD 及其在 RS 中的创新应用进行了广泛的探讨。KD 是一种将知识从复杂、繁琐的模型（教师）转移到更紧凑、更高效的模型（学生）的技术，在各个领域都有显著的发展和应用。首先，我们将介绍 KD 方法的基本概念和历史进程。文章强调了采用 KD 的优势，特别是在模型压缩、提高计算效率和改善性能方面，这些优势对于 RS 场景中的实际部署至关重要。文章对 KD 技术进行了全面分类，对每一类技术都进行了批判性分析，以展示备选方案的广度和深度，并通过具体案例研究展示了 KD 方法在 RS 任务（如实例分割和对象检测）中的实际应用。此外，综述还讨论了 KD 在 RS 中面临的挑战和局限性，包括实际限制因素和未来发展方向，为 RS 领域的研究人员和从业人员提供了一个全面的概览。通过这样的组织，本文不仅阐明了 KD 的研究现状，还为未来的研究机会奠定了基础，从而为学术研究和实际应用做出了重要贡献。

{"title":"Applications of knowledge distillation in remote sensing: A survey","authors":"Yassine Himeur , Nour Aburaed , Omar Elharrouss , Iraklis Varlamis , Shadi Atalla , Wathiq Mansoor , Hussain Al-Ahmad","doi":"10.1016/j.inffus.2024.102742","DOIUrl":"10.1016/j.inffus.2024.102742","url":null,"abstract":"<div><div>With the ever-growing complexity of models in the field of remote sensing (RS), there is an increasing demand for solutions that balance model accuracy with computational efficiency. Knowledge distillation (KD) has emerged as a powerful tool to meet this need, enabling the transfer of knowledge from large, complex models to smaller, more efficient ones without significant loss in performance. This review article provides an extensive examination of KD and its innovative applications in RS. KD, a technique developed to transfer knowledge from a complex, often cumbersome model (teacher) to a more compact and efficient model (student), has seen significant evolution and application across various domains. Initially, we introduce the fundamental concepts and historical progression of KD methods. The advantages of employing KD are highlighted, particularly in terms of model compression, enhanced computational efficiency, and improved performance, which are pivotal for practical deployments in RS scenarios. The article provides a comprehensive taxonomy of KD techniques, where each category is critically analyzed to demonstrate the breadth and depth of the alternative options, and illustrates specific case studies that showcase the practical implementation of KD methods in RS tasks, such as instance segmentation and object detection. Further, the review discusses the challenges and limitations of KD in RS, including practical constraints and prospective future directions, providing a comprehensive overview for researchers and practitioners in the field of RS. Through this organization, the paper not only elucidates the current state of research in KD but also sets the stage for future research opportunities, thereby contributing significantly to both academic research and real-world applications.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"115 ","pages":"Article 102742"},"PeriodicalIF":14.7,"publicationDate":"2024-10-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142554838","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Label distribution-driven multi-view representation learning 标签分布驱动的多视图表示学习

IF 14.7 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2024-10-19 DOI: 10.1016/j.inffus.2024.102727

Wenbiao Yan , Minghong Wu , Yiyang Zhou , Qinghai Zheng , Jinqian Chen , Haozhe Cheng , Jihua Zhu

In multi-view representation learning (MVRL), the challenge of category uncertainty is significant. Existing methods excel at deriving shared representations across multiple views, but often neglect the uncertainty associated with cluster assignments from each view, thereby leading to increased ambiguity in the category determination. Additionally, methods like kernel-based or neural network-based approaches, while revealing nonlinear relationships, lack attention to category uncertainty. To address these limitations, this paper proposes a method leveraging the uncertainty of label distributions to enhance MVRL. Specifically, our approach combines uncertainty reduction based on label distribution with view representation learning to improve clustering accuracy and robustness. It initially computes the within-view representation of the sample and semantic labels. Then, we introduce a novel constraint based on either variance or information entropy to mitigate class uncertainty, thereby improving the discriminative power of the learned representations. Extensive experiments conducted on diverse multi-view datasets demonstrate that our method consistently outperforms existing approaches, producing more accurate and reliable class assignments. The experimental results highlight the effectiveness of our method in enhancing MVRL by reducing category uncertainty and improving overall classification performance. This method is not only very interpretable but also enhances the model’s ability to learn multi-view consistent information.

在多视图表征学习（MVRL）中，类别不确定性是一个重大挑战。现有的方法擅长在多个视图中推导出共享表征，但往往忽略了与每个视图的聚类分配相关的不确定性，从而导致类别确定的模糊性增加。此外，基于内核或神经网络的方法虽然能揭示非线性关系，但缺乏对类别不确定性的关注。为了解决这些局限性，本文提出了一种利用标签分布的不确定性来增强 MVRL 的方法。具体来说，我们的方法将基于标签分布的不确定性降低与视图表示学习相结合，以提高聚类的准确性和鲁棒性。它首先计算样本和语义标签的视图内表示。然后，我们引入基于方差或信息熵的新颖约束来减少类的不确定性，从而提高所学表征的判别能力。在各种多视图数据集上进行的广泛实验表明，我们的方法始终优于现有方法，能产生更准确、更可靠的类别分配。实验结果凸显了我们的方法在通过减少类别不确定性和提高整体分类性能来增强 MVRL 方面的有效性。这种方法不仅具有很强的可解释性，而且还增强了模型学习多视角一致信息的能力。

{"title":"Label distribution-driven multi-view representation learning","authors":"Wenbiao Yan , Minghong Wu , Yiyang Zhou , Qinghai Zheng , Jinqian Chen , Haozhe Cheng , Jihua Zhu","doi":"10.1016/j.inffus.2024.102727","DOIUrl":"10.1016/j.inffus.2024.102727","url":null,"abstract":"<div><div>In multi-view representation learning (MVRL), the challenge of category uncertainty is significant. Existing methods excel at deriving shared representations across multiple views, but often neglect the uncertainty associated with cluster assignments from each view, thereby leading to increased ambiguity in the category determination. Additionally, methods like kernel-based or neural network-based approaches, while revealing nonlinear relationships, lack attention to category uncertainty. To address these limitations, this paper proposes a method leveraging the uncertainty of label distributions to enhance MVRL. Specifically, our approach combines uncertainty reduction based on label distribution with view representation learning to improve clustering accuracy and robustness. It initially computes the within-view representation of the sample and semantic labels. Then, we introduce a novel constraint based on either variance or information entropy to mitigate class uncertainty, thereby improving the discriminative power of the learned representations. Extensive experiments conducted on diverse multi-view datasets demonstrate that our method consistently outperforms existing approaches, producing more accurate and reliable class assignments. The experimental results highlight the effectiveness of our method in enhancing MVRL by reducing category uncertainty and improving overall classification performance. This method is not only very interpretable but also enhances the model’s ability to learn multi-view consistent information.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"115 ","pages":"Article 102727"},"PeriodicalIF":14.7,"publicationDate":"2024-10-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142554834","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

HSMix: Hard and soft mixing data augmentation for medical image segmentation HSMix：用于医学图像分割的软硬混合数据增强技术

IF 14.7 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2024-10-18 DOI: 10.1016/j.inffus.2024.102741

D. Sun , F. Dornaika , N. Barrena

Due to the high cost of annotation or the rarity of some diseases, medical image segmentation is often limited by data scarcity and the resulting overfitting problem. Self-supervised learning and semi-supervised learning can mitigate the data scarcity challenge to some extent. However, both of these paradigms are complex and require either hand-crafted pretexts or well-defined pseudo-labels. In contrast, data augmentation represents a relatively simple and straightforward approach to addressing data scarcity issues. It has led to significant improvements in image recognition tasks. However, the effectiveness of local image editing augmentation techniques in the context of segmentation has been less explored. Additionally, traditional data augmentation methods for local image editing augmentation methods generally utilize square regions, which cause a loss of contour information.

We propose HSMix, a novel approach to local image editing data augmentation involving hard and soft mixing for medical semantic segmentation. In our approach, a hard-augmented image is created by combining homogeneous regions (superpixels) from two source images. A soft mixing method further adjusts the brightness of these composed regions with brightness mixing based on locally aggregated pixel-wise saliency coefficients. The ground-truth segmentation masks of the two source images undergo the same mixing operations to generate the associated masks for the augmented images.

Our method fully exploits both the prior contour and saliency information, thus preserving local semantic information in the augmented images while enriching the augmentation space with more diversity. Our method is a plug-and-play solution that is model agnostic and applicable to a range of medical imaging modalities. Extensive experimental evidence has demonstrated its effectiveness in a variety of medical segmentation tasks. The source code is available in https://github.com/DanielaPlusPlus/HSMix.

由于标注成本高昂或某些疾病的罕见性，医学影像分割往往受限于数据稀缺以及由此产生的过拟合问题。自监督学习和半监督学习可以在一定程度上缓解数据稀缺的难题。不过，这两种模式都很复杂，需要手工制作的借口或定义明确的伪标签。相比之下，数据增强是解决数据稀缺问题的一种相对简单直接的方法。它极大地改进了图像识别任务。然而，局部图像编辑增强技术在分割方面的有效性却鲜有人问津。此外，用于局部图像编辑增强方法的传统数据增强方法通常使用正方形区域，这会导致轮廓信息的丢失。我们提出了 HSMix，这是一种新颖的局部图像编辑数据增强方法，涉及医疗语义分割中的软硬混合。在我们的方法中，通过将两个源图像中的同质区域（超像素）组合在一起，创建硬增强图像。软混合方法根据局部聚集的像素显著性系数，通过亮度混合进一步调整这些组成区域的亮度。我们的方法充分利用了先验轮廓信息和显著性信息，从而保留了增强图像中的局部语义信息，同时丰富了增强空间的多样性。我们的方法是一种即插即用的解决方案，与模型无关，适用于各种医学成像模式。广泛的实验证明了它在各种医学分割任务中的有效性。源代码见 https://github.com/DanielaPlusPlus/HSMix。

{"title":"HSMix: Hard and soft mixing data augmentation for medical image segmentation","authors":"D. Sun , F. Dornaika , N. Barrena","doi":"10.1016/j.inffus.2024.102741","DOIUrl":"10.1016/j.inffus.2024.102741","url":null,"abstract":"<div><div>Due to the high cost of annotation or the rarity of some diseases, medical image segmentation is often limited by data scarcity and the resulting overfitting problem. Self-supervised learning and semi-supervised learning can mitigate the data scarcity challenge to some extent. However, both of these paradigms are complex and require either hand-crafted pretexts or well-defined pseudo-labels. In contrast, data augmentation represents a relatively simple and straightforward approach to addressing data scarcity issues. It has led to significant improvements in image recognition tasks. However, the effectiveness of local image editing augmentation techniques in the context of segmentation has been less explored. Additionally, traditional data augmentation methods for local image editing augmentation methods generally utilize square regions, which cause a loss of contour information.</div><div>We propose HSMix, a novel approach to local image editing data augmentation involving hard and soft mixing for medical semantic segmentation. In our approach, a hard-augmented image is created by combining homogeneous regions (superpixels) from two source images. A soft mixing method further adjusts the brightness of these composed regions with brightness mixing based on locally aggregated pixel-wise saliency coefficients. The ground-truth segmentation masks of the two source images undergo the same mixing operations to generate the associated masks for the augmented images.</div><div>Our method fully exploits both the prior contour and saliency information, thus preserving local semantic information in the augmented images while enriching the augmentation space with more diversity. Our method is a plug-and-play solution that is model agnostic and applicable to a range of medical imaging modalities. Extensive experimental evidence has demonstrated its effectiveness in a variety of medical segmentation tasks. The source code is available in <span><span>https://github.com/DanielaPlusPlus/HSMix</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"115 ","pages":"Article 102741"},"PeriodicalIF":14.7,"publicationDate":"2024-10-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142531905","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

An adaptive meta-imitation learning-based recommendation environment simulator: A case study on ship-cargo matching 基于元模仿学习的自适应推荐环境模拟器：船货匹配案例研究

IF 14.7 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2024-10-18 DOI: 10.1016/j.inffus.2024.102740

Guangyao Pang , Jiehang Xie , Fei Hao

High-quality shipping is one of the effective ways for sustainable cities in inland river basins to improve transportation efficiency and reduce energy consumption. Currently, the biggest challenge faced by shipping is the high empty-ship rate, which makes it impossible to directly apply machine learning methods due to the cold-start problem. Although some researchers have tried to utilize deep reinforcement learning(DRL)-based recommendation that do not rely on manually labeled data to alleviate the cold-start problem, progress has been slow due to the lack of available training environment. Therefore, this paper introduces an adaptive meta-imitation learning-based recommendation environment simulator, termed AMIL-Simulator. Specifically, we construct a conditionally guided diffusion model to simulate shipowner behavior in a dynamically changing environment. Moreover, we propose a shipowner reward model based on adaptive meta-imitation learning, enabling the learning of shipowner rewards across multiple tasks, even when confronted with limited samples and imbalanced categories. By conducting extensive quantitative experimental evaluations and shipowner-cargo matching studies, the results demonstrate the effectiveness of AMIL-Simulator, particularly in smaller-scale and cold-start environments.

高质量的航运是内河流域可持续发展城市提高运输效率、降低能源消耗的有效途径之一。目前，航运业面临的最大挑战是空船率高，由于冷启动问题，无法直接应用机器学习方法。虽然一些研究人员已经尝试利用基于深度强化学习（DRL）的推荐方法来缓解冷启动问题，但由于缺乏可用的训练环境，进展十分缓慢。因此，本文介绍了一种基于元模仿学习的自适应推荐环境模拟器，称为 AMIL-Simulator。具体来说，我们构建了一个条件引导扩散模型，以模拟动态变化环境中的船东行为。此外，我们还提出了基于自适应元模仿学习的船东奖励模型，即使面对有限的样本和不平衡的类别，也能在多个任务中学习船东奖励。通过广泛的定量实验评估和船东-货物匹配研究，结果证明了 AMIL 模拟器的有效性，尤其是在较小规模和冷启动环境中。

{"title":"An adaptive meta-imitation learning-based recommendation environment simulator: A case study on ship-cargo matching","authors":"Guangyao Pang , Jiehang Xie , Fei Hao","doi":"10.1016/j.inffus.2024.102740","DOIUrl":"10.1016/j.inffus.2024.102740","url":null,"abstract":"<div><div>High-quality shipping is one of the effective ways for sustainable cities in inland river basins to improve transportation efficiency and reduce energy consumption. Currently, the biggest challenge faced by shipping is the high empty-ship rate, which makes it impossible to directly apply machine learning methods due to the cold-start problem. Although some researchers have tried to utilize deep reinforcement learning(DRL)-based recommendation that do not rely on manually labeled data to alleviate the cold-start problem, progress has been slow due to the lack of available training environment. Therefore, this paper introduces an adaptive meta-imitation learning-based recommendation environment simulator, termed AMIL-Simulator. Specifically, we construct a conditionally guided diffusion model to simulate shipowner behavior in a dynamically changing environment. Moreover, we propose a shipowner reward model based on adaptive meta-imitation learning, enabling the learning of shipowner rewards across multiple tasks, even when confronted with limited samples and imbalanced categories. By conducting extensive quantitative experimental evaluations and shipowner-cargo matching studies, the results demonstrate the effectiveness of AMIL-Simulator, particularly in smaller-scale and cold-start environments.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"115 ","pages":"Article 102740"},"PeriodicalIF":14.7,"publicationDate":"2024-10-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142531896","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Recent advances in data-driven fusion of multi-modal imaging and genomics for precision medicine 数据驱动的多模态成像与基因组学融合促进精准医疗的最新进展

IF 14.7 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2024-10-16 DOI: 10.1016/j.inffus.2024.102738

Shuo Wang , Meng Liu , Yan Li , Xinyu Zhang , Mengting Sun , Zian Wang , Ruokun Li , Qirong Li , Qing Li , Yili He , Xumei Hu , Longyu Sun , Fuhua Yan , Mengyao Yu , Weiping Ding , Chengyan Wang

Imaging genomics is poised to revolutionize clinical practice by providing deep insights into the genetic underpinnings of disease, enabling early detection, and facilitating personalized treatment strategies. The field has seen remarkable advancements, with significant momentum fueled by cutting-edge imaging techniques, sophisticated data-driven fusion methods, and extensive large cohort datasets. Originally centered on the brain, imaging genomics has now expanded to encompass other organs throughout the body. Due to the highly interdisciplinary nature involving medical imaging, genetics, machine learning, and clinical medicine, readers who wish to conduct research in this field urgently need a comprehensive review. This survey provides an overview of recent advancements in data-driven fusion of multi-modal imaging and genomics, covering applications in the brain, heart, lungs, breasts, abdomen, and bones. We summarize three primary fusion strategies: correlation analysis, causal analysis, and machine learning, discussing their respective application scenarios. Additionally, we explore clinical applications that integrate imaging datasets and genomic data across six major organ systems, and present available open datasets featuring both modalities. Finally, we summarize the challenges and future directions in imaging genomics, which include improving data representation, integrating other omics data, conducting cross-dataset analyses, advancing machine learning algorithms, and investigating organ interactions. This survey aims to review the latest developments in data-driven fusion for precision medicine while providing insights into the future of this evolving field.

成像基因组学能够深入洞察疾病的遗传基础，实现早期检测，并促进个性化治疗策略，有望彻底改变临床实践。在尖端成像技术、复杂的数据驱动融合方法和广泛的大型队列数据集的推动下，该领域取得了令人瞩目的进展。成像基因组学最初以大脑为中心，现在已扩展到全身其他器官。由于成像基因组学涉及医学成像、遗传学、机器学习和临床医学等多个学科，具有高度的跨学科性质，因此希望在这一领域开展研究的读者迫切需要一本全面的综述。本研究概述了数据驱动的多模态成像与基因组学融合的最新进展，涵盖大脑、心脏、肺部、乳房、腹部和骨骼的应用。我们总结了三种主要的融合策略：相关分析、因果分析和机器学习，并讨论了它们各自的应用场景。此外，我们还探讨了在六大器官系统中整合成像数据集和基因组数据的临床应用，并介绍了以这两种模式为特色的可用开放数据集。最后，我们总结了成像基因组学面临的挑战和未来发展方向，其中包括改进数据表示、整合其他 omics 数据、进行跨数据集分析、推进机器学习算法以及研究器官相互作用。本调查旨在回顾数据驱动的精准医学融合的最新发展，同时为这一不断发展的领域的未来提供见解。

{"title":"Recent advances in data-driven fusion of multi-modal imaging and genomics for precision medicine","authors":"Shuo Wang , Meng Liu , Yan Li , Xinyu Zhang , Mengting Sun , Zian Wang , Ruokun Li , Qirong Li , Qing Li , Yili He , Xumei Hu , Longyu Sun , Fuhua Yan , Mengyao Yu , Weiping Ding , Chengyan Wang","doi":"10.1016/j.inffus.2024.102738","DOIUrl":"10.1016/j.inffus.2024.102738","url":null,"abstract":"<div><div>Imaging genomics is poised to revolutionize clinical practice by providing deep insights into the genetic underpinnings of disease, enabling early detection, and facilitating personalized treatment strategies. The field has seen remarkable advancements, with significant momentum fueled by cutting-edge imaging techniques, sophisticated data-driven fusion methods, and extensive large cohort datasets. Originally centered on the brain, imaging genomics has now expanded to encompass other organs throughout the body. Due to the highly interdisciplinary nature involving medical imaging, genetics, machine learning, and clinical medicine, readers who wish to conduct research in this field urgently need a comprehensive review. This survey provides an overview of recent advancements in data-driven fusion of multi-modal imaging and genomics, covering applications in the brain, heart, lungs, breasts, abdomen, and bones. We summarize three primary fusion strategies: correlation analysis, causal analysis, and machine learning, discussing their respective application scenarios. Additionally, we explore clinical applications that integrate imaging datasets and genomic data across six major organ systems, and present available open datasets featuring both modalities. Finally, we summarize the challenges and future directions in imaging genomics, which include improving data representation, integrating other omics data, conducting cross-dataset analyses, advancing machine learning algorithms, and investigating organ interactions. This survey aims to review the latest developments in data-driven fusion for precision medicine while providing insights into the future of this evolving field.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"115 ","pages":"Article 102738"},"PeriodicalIF":14.7,"publicationDate":"2024-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142531902","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

An interactive iteration consensus based social network large-scale group decision making method and its application in zero-waste city evaluation 基于互动迭代共识的社会网络大规模群体决策方法及其在零废弃物城市评估中的应用

IF 14.7 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2024-10-16 DOI: 10.1016/j.inffus.2024.102744

Fanyong Meng , Hao Li , Jinyu Li

The construction of zero-waste (ZW) cities receives increasing attention from the Chinese government. The evaluation is essential to make policy variations according to the actual situation in each place. Previous assessments of ZW cities have primarily relied on historical data, which fails to account for the subjective preferences of various stakeholders. For example, it is challenging to capture residents' subjective opinions about the development of a ZW city. This paper presents a social network large-scale group decision-making method for evaluating the construction of ZW city. First, experts' evaluation opinions and trust relations are used to develop an improved clustering method. The weights of the clusters are then determined using internal-external cohesion indices and the number of experts, with experts' weights defined by their similarity-trust degree. An optimization model based on interactive iteration consensus is formulated, considering the fairness and rationality of allocation schemes. Additionally, a new social network large-scale group decision-making method is presented. Finally, the proposed method is illustrated with a case study of selecting a national-level ZW city in Jiangsu Province.

零废弃物（ZW）城市的建设越来越受到中国政府的重视。要根据各地的实际情况进行政策调整，评估必不可少。以往的零废弃城市评估主要依赖于历史数据，未能考虑各利益相关方的主观偏好。例如，捕捉居民对 ZW 城市发展的主观意见是一项挑战。本文提出了一种评估 ZW 城市建设的社会网络大规模群体决策方法。首先，利用专家的评价意见和信任关系开发了一种改进的聚类方法。然后利用内外部凝聚力指数和专家人数确定聚类的权重，专家权重由其相似度-信任度定义。考虑到分配方案的公平性和合理性，建立了基于交互迭代共识的优化模型。此外，还提出了一种新的社会网络大规模群体决策方法。最后，以江苏省选择国家级 ZW 城市的案例研究说明了所提出的方法。

{"title":"An interactive iteration consensus based social network large-scale group decision making method and its application in zero-waste city evaluation","authors":"Fanyong Meng , Hao Li , Jinyu Li","doi":"10.1016/j.inffus.2024.102744","DOIUrl":"10.1016/j.inffus.2024.102744","url":null,"abstract":"<div><div>The construction of zero-waste (ZW) cities receives increasing attention from the Chinese government. The evaluation is essential to make policy variations according to the actual situation in each place. Previous assessments of ZW cities have primarily relied on historical data, which fails to account for the subjective preferences of various stakeholders. For example, it is challenging to capture residents' subjective opinions about the development of a ZW city. This paper presents a social network large-scale group decision-making method for evaluating the construction of ZW city. First, experts' evaluation opinions and trust relations are used to develop an improved clustering method. The weights of the clusters are then determined using internal-external cohesion indices and the number of experts, with experts' weights defined by their similarity-trust degree. An optimization model based on interactive iteration consensus is formulated, considering the fairness and rationality of allocation schemes. Additionally, a new social network large-scale group decision-making method is presented. Finally, the proposed method is illustrated with a case study of selecting a national-level ZW city in Jiangsu Province.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"115 ","pages":"Article 102744"},"PeriodicalIF":14.7,"publicationDate":"2024-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142531897","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Multi-view support vector machine classifier via L0/1 soft-margin loss with structural information 通过具有结构信息的 L0/1 软边际损失实现多视角支持向量机分类器

IF 14.7 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2024-10-16 DOI: 10.1016/j.inffus.2024.102733

Chen Chen , Qianfei Liu , Renpeng Xu , Ying Zhang , Huiru Wang , Qingmin Yu

Multi-view learning seeks to leverage the advantages of various views to complement each other and make full use of the latent information in the data. Nevertheless, effectively exploring and utilizing common and complementary information across diverse views remains challenging. In this paper, we propose two multi-view classifiers: multi-view support vector machine via

L_{0 / 1}

soft-margin loss (Mv

L_{0 / 1}

-SVM) and structural Mv

L_{0 / 1}

-SVM (Mv

S L_{0 / 1}

-SVM). The key difference between them is that Mv

S L_{0 / 1}

-SVM additionally fuses structural information, which simultaneously satisfies the consensus and complementarity principles. Despite the discrete nature inherent in the

L_{0 / 1}

soft-margin loss, we successfully establish the optimality theory for Mv

S L_{0 / 1}

-SVM. This includes demonstrating the existence of optimal solutions and elucidating their relationships with P-stationary points. Drawing inspiration from the P-stationary point optimality condition, we design and integrate a working set strategy into the proximal alternating direction method of multipliers. This integration significantly enhances the overall computational speed and diminishes the number of support vectors. Last but not least, numerical experiments show that our suggested models perform exceptionally well and have faster computational speed, affirming the rationality and effectiveness of our methods.

多视图学习旨在利用各种视图的优势，取长补短，充分利用数据中的潜在信息。然而，有效地探索和利用不同视图之间的共同和互补信息仍是一项挑战。本文提出了两种多视图分类器：通过 L0/1 软边际损失的多视图支持向量机（MvL0/1-SVM）和结构 MvL0/1-SVM （MvSL0/1-SVM）。它们之间的主要区别在于，MvSL0/1-SVM 还融合了结构信息，同时满足共识和互补原则。尽管 L0/1 软边际损失具有固有的离散性，我们还是成功地建立了 MvSL0/1-SVM 的最优性理论。这包括证明最优解的存在，并阐明它们与 P-stationary 点的关系。从 P-stationary 点最优条件中汲取灵感，我们设计了一种工作集策略，并将其集成到近似交替方向乘法中。这种整合大大提高了整体计算速度，并减少了支持向量的数量。最后但并非最不重要的一点是，数值实验表明，我们建议的模型性能优异，计算速度更快，从而肯定了我们方法的合理性和有效性。

{"title":"Multi-view support vector machine classifier via L0/1 soft-margin loss with structural information","authors":"Chen Chen , Qianfei Liu , Renpeng Xu , Ying Zhang , Huiru Wang , Qingmin Yu","doi":"10.1016/j.inffus.2024.102733","DOIUrl":"10.1016/j.inffus.2024.102733","url":null,"abstract":"<div><div>Multi-view learning seeks to leverage the advantages of various views to complement each other and make full use of the latent information in the data. Nevertheless, effectively exploring and utilizing common and complementary information across diverse views remains challenging. In this paper, we propose two multi-view classifiers: multi-view support vector machine via <span><math><msub><mrow><mi>L</mi></mrow><mrow><mn>0</mn><mo>/</mo><mn>1</mn></mrow></msub></math></span> soft-margin loss (Mv<span><math><msub><mrow><mi>L</mi></mrow><mrow><mn>0</mn><mo>/</mo><mn>1</mn></mrow></msub></math></span>-SVM) and structural Mv<span><math><msub><mrow><mi>L</mi></mrow><mrow><mn>0</mn><mo>/</mo><mn>1</mn></mrow></msub></math></span>-SVM (Mv<span><math><mrow><mi>S</mi><msub><mrow><mi>L</mi></mrow><mrow><mn>0</mn><mo>/</mo><mn>1</mn></mrow></msub></mrow></math></span>-SVM). The key difference between them is that Mv<span><math><mrow><mi>S</mi><msub><mrow><mi>L</mi></mrow><mrow><mn>0</mn><mo>/</mo><mn>1</mn></mrow></msub></mrow></math></span>-SVM additionally fuses structural information, which simultaneously satisfies the consensus and complementarity principles. Despite the discrete nature inherent in the <span><math><msub><mrow><mi>L</mi></mrow><mrow><mn>0</mn><mo>/</mo><mn>1</mn></mrow></msub></math></span> soft-margin loss, we successfully establish the optimality theory for Mv<span><math><mrow><mi>S</mi><msub><mrow><mi>L</mi></mrow><mrow><mn>0</mn><mo>/</mo><mn>1</mn></mrow></msub></mrow></math></span>-SVM. This includes demonstrating the existence of optimal solutions and elucidating their relationships with P-stationary points. Drawing inspiration from the P-stationary point optimality condition, we design and integrate a working set strategy into the proximal alternating direction method of multipliers. This integration significantly enhances the overall computational speed and diminishes the number of support vectors. Last but not least, numerical experiments show that our suggested models perform exceptionally well and have faster computational speed, affirming the rationality and effectiveness of our methods.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"115 ","pages":"Article 102733"},"PeriodicalIF":14.7,"publicationDate":"2024-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142531901","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Open knowledge graph completion with negative-aware representation learning and multi-source reliability inference 利用负面感知表征学习和多源可靠性推理完成开放式知识图谱

IF 14.7 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2024-10-16 DOI: 10.1016/j.inffus.2024.102729

Huang Peng, Weixin Zeng, Jiuyang Tang, Mao Wang, Hongbin Huang, Xiang Zhao

Multi-source data fusion is essential for building smart cities by providing a comprehensive and holistic understanding of urban environments. Specifically, smart city-oriented knowledge graphs (KGs) require supplementary information from other open sources to increase their completeness, thus better supporting downstream tasks for smart cities. Nevertheless, existing open knowledge graph completion (KGC) approaches often overlook source quality assessment and fail to fully utilize prior knowledge, which tend to yield less satisfying results. To fill in these gaps, in this work, we propose a new open KGC method with negative-aware representation learning and multi-source reliability inference, i.e., Nari, which can effectively integrate the multi-source data concerning sustainable cities, providing reliable knowledge for downstream tasks. Specifically, we first train a graph neural network based encoder with a novel negative sampling strategy to better characterize prior knowledge in KG, and then identify new facts based on the learned prior knowledge and source reliability. The experiments on both general benchmark and waterlogging benchmark pertaining to sustainable cities demonstrate the effectiveness and wide applicability of Nari.

多源数据融合可提供对城市环境的全面整体理解，对于建设智慧城市至关重要。具体来说，面向智慧城市的知识图谱（KG）需要来自其他开放源的补充信息来提高其完整性，从而更好地支持智慧城市的下游任务。然而，现有的开放式知识图谱补全（KGC）方法往往忽略了源质量评估，也未能充分利用先验知识，因此往往无法获得令人满意的结果。为了填补这些空白，我们在这项工作中提出了一种具有负感知表征学习和多源可靠性推理的新型开放式知识图谱方法，即 Nari，它能有效整合有关可持续城市的多源数据，为下游任务提供可靠的知识。具体来说，我们首先使用新颖的负采样策略训练基于图神经网络的编码器，以更好地表征 KG 中的先验知识，然后根据学习到的先验知识和来源可靠性识别新事实。在一般基准和与可持续城市相关的内涝基准上进行的实验证明了 Nari 的有效性和广泛适用性。

{"title":"Open knowledge graph completion with negative-aware representation learning and multi-source reliability inference","authors":"Huang Peng, Weixin Zeng, Jiuyang Tang, Mao Wang, Hongbin Huang, Xiang Zhao","doi":"10.1016/j.inffus.2024.102729","DOIUrl":"10.1016/j.inffus.2024.102729","url":null,"abstract":"<div><div>Multi-source data fusion is essential for building smart cities by providing a comprehensive and holistic understanding of urban environments. Specifically, smart city-oriented knowledge graphs (KGs) require supplementary information from other open sources to increase their completeness, thus better supporting downstream tasks for smart cities. Nevertheless, existing open knowledge graph completion (KGC) approaches often overlook source quality assessment and fail to fully utilize prior knowledge, which tend to yield less satisfying results. To fill in these gaps, in this work, we propose a new open KGC method with negative-aware representation learning and multi-source reliability inference, i.e., <span>Nari</span>, which can effectively integrate the multi-source data concerning sustainable cities, providing reliable knowledge for downstream tasks. Specifically, we first train a graph neural network based encoder with a novel negative sampling strategy to better characterize prior knowledge in KG, and then identify new facts based on the learned prior knowledge and source reliability. The experiments on both general benchmark and waterlogging benchmark pertaining to sustainable cities demonstrate the effectiveness and wide applicability of <span>Nari</span>.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"115 ","pages":"Article 102729"},"PeriodicalIF":14.7,"publicationDate":"2024-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142531892","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Blockchain-based privacy-preserving incentive scheme for internet of electric vehicle 基于区块链的电动汽车互联网隐私保护激励方案

IF 14.7 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2024-10-16 DOI: 10.1016/j.inffus.2024.102732

Qian Mei , Wenxia Guo , Yanan Zhao , Liming Nie , Deepak Adhikari

The emerging proportion of renewable energy resources penetration and the rapid popularity of Electric Vehicles (EVs) have promoted the development of the Internet of Electric Vehicles (IoEV), which enables seamless EV’ information collection and energy delivery by leveraging wireless power transfer. However, vulnerabilities in internet infrastructure and the self-interested behavior of EVs pose significant security and privacy risks during energy delivery in IoEV. In addition, EVs often lack the incentive to cooperate for regional energy balance. To tackle these questions, this paper proposes a blockchain-based privacy-preserving incentive mechanism for energy delivery in IoEV. Based on cryptographic technology, this paper introduces a group signature scheme with self-controlled and sequential linkability, which safeguards the privacy of EV users and ensures transaction records maintain exact sequence during energy delivery. Furthermore, an incentive mechanism based on co-utile reputation management is presented to encourage EV users to participate honestly and cooperatively in energy delivery. Moreover, a comprehensive security analysis of the proposed group signature scheme and incentive mechanism is given. Finally, extensive experimental results demonstrate the feasibility and efficiency of the proposed approach compared to existing schemes.

可再生能源渗透率的不断提高和电动汽车（EV）的迅速普及促进了电动汽车互联网（IoEV）的发展，它通过利用无线电力传输实现了无缝的电动汽车信息收集和能源传输。然而，互联网基础设施的漏洞和电动汽车的自利行为在 IoEV 能量传输过程中带来了巨大的安全和隐私风险。此外，电动汽车往往缺乏合作实现区域能源平衡的动力。为解决这些问题，本文提出了一种基于区块链的物联网能源交付隐私保护激励机制。基于加密技术，本文引入了一种具有自控和顺序链接性的群签名方案，既能保护电动汽车用户的隐私，又能确保交易记录在能源交付过程中保持准确的顺序。此外，本文还提出了一种基于共同利益声誉管理的激励机制，以鼓励电动汽车用户诚实、合作地参与能源交付。此外，还对所提出的群组签名方案和激励机制进行了全面的安全分析。最后，大量实验结果表明，与现有方案相比，建议的方法既可行又高效。

{"title":"Blockchain-based privacy-preserving incentive scheme for internet of electric vehicle","authors":"Qian Mei , Wenxia Guo , Yanan Zhao , Liming Nie , Deepak Adhikari","doi":"10.1016/j.inffus.2024.102732","DOIUrl":"10.1016/j.inffus.2024.102732","url":null,"abstract":"<div><div>The emerging proportion of renewable energy resources penetration and the rapid popularity of Electric Vehicles (EVs) have promoted the development of the Internet of Electric Vehicles (IoEV), which enables seamless EV’ information collection and energy delivery by leveraging wireless power transfer. However, vulnerabilities in internet infrastructure and the self-interested behavior of EVs pose significant security and privacy risks during energy delivery in IoEV. In addition, EVs often lack the incentive to cooperate for regional energy balance. To tackle these questions, this paper proposes a blockchain-based privacy-preserving incentive mechanism for energy delivery in IoEV. Based on cryptographic technology, this paper introduces a group signature scheme with self-controlled and sequential linkability, which safeguards the privacy of EV users and ensures transaction records maintain exact sequence during energy delivery. Furthermore, an incentive mechanism based on co-utile reputation management is presented to encourage EV users to participate honestly and cooperatively in energy delivery. Moreover, a comprehensive security analysis of the proposed group signature scheme and incentive mechanism is given. Finally, extensive experimental results demonstrate the feasibility and efficiency of the proposed approach compared to existing schemes.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"115 ","pages":"Article 102732"},"PeriodicalIF":14.7,"publicationDate":"2024-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142531895","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

An efficient cross-view image fusion method based on selected state space and hashing for promoting urban perception 基于选定状态空间和散列的高效跨视角图像融合方法促进城市感知

IF 14.7 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2024-10-15 DOI: 10.1016/j.inffus.2024.102737

Peng Han , Chao Chen

In the field of cross-view image geolocation, traditional convolutional neural network (CNN)-based learning models generate unsatisfactory fusion performance due to their inability to model global correlations. The Transformer-based fusion methods can well compensate for the above problems, however, the Transformer has quadratic computational complexity and huge GPU memory consumption. The recent Mamba model based on the selection state space has a strong ability to model long sequences, lower GPU memory occupancy, and fewer GFLOPs. It is thus attractive and worth studying to apply Mamba to the cross-view image geolocation task. In addition, in the image-matching process (i.e., fusion of satellite/aerial and street view data.), we found that the storage occupancy of similarity measures based on floating-point features is high. Efficiently converting floating-point features into hash codes is a possible solution. In this study, we propose a cross-view image geolocation method (S6HG) based purely on Vision Mamba and hashing. S6HG fully utilizes the advantages of Vision Mamba in global information modeling and explicit location information encoding and the low storage occupancy of hash codes. Our method consists of two stages. In the first stage, we use a Siamese network based purely on vision Mamba to embed features for street view images and satellite images respectively. Our first-stage model is called S6G. In the second stage, we construct a cross-view autoencoder to further refine and compress the embedded features, and then simply map the refined features to hash codes. Comprehensive experiments show that S6G has achieved superior results on the CVACT dataset and comparable results to the most advanced methods on the CVUSA dataset. It is worth noting that other floating-point feature-based methods (4096-dimension) are 170.59 times faster than S6HG (768-bit) in storing 90,618 retrieval gallery data. Furthermore, the inference efficiency of S6G is higher than ViT-based computational methods.

在跨视角图像地理定位领域，传统的基于卷积神经网络（CNN）的学习模型由于无法建立全局相关性模型，其融合性能并不理想。基于变换器的融合方法可以很好地弥补上述问题，但变换器具有二次计算复杂性和巨大的 GPU 内存消耗。最近推出的基于选择状态空间的 Mamba 模型具有很强的长序列建模能力、较低的 GPU 内存占用和较少的 GFLOP。因此，将 Mamba 应用于跨视角图像地理定位任务是非常有吸引力和值得研究的。此外，在图像匹配过程（即卫星/航拍和街景数据的融合）中，我们发现基于浮点特征的相似性度量的存储占用率很高。有效地将浮点特征转换成哈希代码是一个可行的解决方案。在本研究中，我们提出了一种纯粹基于 Vision Mamba 和散列的跨视图图像地理定位方法（S6HG）。S6HG 充分利用了 Vision Mamba 在全局信息建模和显式位置信息编码方面的优势，以及散列码的低存储占用率。我们的方法包括两个阶段。在第一阶段，我们使用纯粹基于视觉 Mamba 的连体网络，分别为街景图像和卫星图像嵌入特征。我们的第一阶段模型称为 S6G。在第二阶段，我们构建了一个跨视图自动编码器来进一步细化和压缩嵌入的特征，然后将细化后的特征简单地映射为哈希代码。综合实验表明，S6G 在 CVACT 数据集上取得了优异的结果，在 CVUSA 数据集上的结果与最先进的方法不相上下。值得注意的是，在存储 90,618 个检索图库数据时，其他基于浮点特征的方法（4096 维）比 S6HG（768 位）快 170.59 倍。此外，S6G 的推理效率也高于基于 ViT 的计算方法。

{"title":"An efficient cross-view image fusion method based on selected state space and hashing for promoting urban perception","authors":"Peng Han , Chao Chen","doi":"10.1016/j.inffus.2024.102737","DOIUrl":"10.1016/j.inffus.2024.102737","url":null,"abstract":"<div><div>In the field of cross-view image geolocation, traditional convolutional neural network (CNN)-based learning models generate unsatisfactory fusion performance due to their inability to model global correlations. The Transformer-based fusion methods can well compensate for the above problems, however, the Transformer has quadratic computational complexity and huge GPU memory consumption. The recent Mamba model based on the selection state space has a strong ability to model long sequences, lower GPU memory occupancy, and fewer GFLOPs. It is thus attractive and worth studying to apply Mamba to the cross-view image geolocation task. In addition, in the image-matching process (i.e., fusion of satellite/aerial and street view data.), we found that the storage occupancy of similarity measures based on floating-point features is high. Efficiently converting floating-point features into hash codes is a possible solution. In this study, we propose a cross-view image geolocation method (S6HG) based purely on Vision Mamba and hashing. S6HG fully utilizes the advantages of Vision Mamba in global information modeling and explicit location information encoding and the low storage occupancy of hash codes. Our method consists of two stages. In the first stage, we use a Siamese network based purely on vision Mamba to embed features for street view images and satellite images respectively. Our first-stage model is called S6G. In the second stage, we construct a cross-view autoencoder to further refine and compress the embedded features, and then simply map the refined features to hash codes. Comprehensive experiments show that S6G has achieved superior results on the CVACT dataset and comparable results to the most advanced methods on the CVUSA dataset. It is worth noting that other floating-point feature-based methods (4096-dimension) are 170.59 times faster than S6HG (768-bit) in storing 90,618 retrieval gallery data. Furthermore, the inference efficiency of S6G is higher than ViT-based computational methods.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"115 ","pages":"Article 102737"},"PeriodicalIF":14.7,"publicationDate":"2024-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142554837","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0