Information Fusion最新文献_第5页

Quantum social network analysis: Methodology, implementation, challenges, and future directions 量子社会网络分析：方法论、实现、挑战和未来方向

IF 14.7 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2024-11-26 DOI: 10.1016/j.inffus.2024.102808

Shashank Sheshar Singh , Sumit Kumar , Sunil Kumar Meena , Kuldeep Singh , Shivansh Mishra , Albert Y. Zomaya

Quantum social network analysis (QSNA) is a recent advancement in the interdisciplinary field of quantum computing and social network analysis. This manuscript comprehensively reviews QSNA, emphasizing its methodologies, implementation strategies, challenges, and potential applications. It explores the conceptual foundation of key social network analysis research problems, including link prediction, influence maximization, and community detection. The research examines how quantum algorithms can revolutionize such social network tasks by leveraging principles from quantum mechanics and information theory and highlights the advantages of quantum algorithms in handling complex social network structures. The implementation section delves into the practical aspects of QSNA, such as frameworks, experimental setups, and evaluation methods. We assess the capabilities of existing quantum programming language tools and platforms. Various case studies illustrate the potential of quantum computing to enhance the performance of social network analysis. Additionally, we identify several crucial challenges and future research directions for QSNA, including the complexity of developing quantum algorithms, the need for interdisciplinary knowledge, and the challenges of integrating quantum and classical computing resources. This paper aims to serve as a foundational resource for researchers and practitioners, providing insights into the transformative potential of quantum computing in advancing the analysis of social networks and outlining future research directions in this emerging field.

量子社会网络分析（QSNA）是量子计算与社会网络分析交叉领域的最新进展。本文全面回顾了QSNA，强调了它的方法、实施策略、挑战和潜在的应用。它探讨了关键社会网络分析研究问题的概念基础，包括链接预测，影响最大化和社区检测。该研究探讨了量子算法如何通过利用量子力学和信息理论的原理来彻底改变这种社会网络任务，并强调了量子算法在处理复杂社会网络结构方面的优势。实现部分将深入探讨QSNA的实际方面，例如框架、实验设置和评估方法。我们评估了现有量子编程语言工具和平台的能力。各种案例研究说明了量子计算在增强社会网络分析性能方面的潜力。此外，我们确定了QSNA的几个关键挑战和未来的研究方向，包括开发量子算法的复杂性，跨学科知识的需求，以及整合量子和经典计算资源的挑战。本文旨在为研究人员和实践者提供基础资源，提供对量子计算在推进社交网络分析方面的变革潜力的见解，并概述这一新兴领域的未来研究方向。

{"title":"Quantum social network analysis: Methodology, implementation, challenges, and future directions","authors":"Shashank Sheshar Singh , Sumit Kumar , Sunil Kumar Meena , Kuldeep Singh , Shivansh Mishra , Albert Y. Zomaya","doi":"10.1016/j.inffus.2024.102808","DOIUrl":"10.1016/j.inffus.2024.102808","url":null,"abstract":"<div><div>Quantum social network analysis (QSNA) is a recent advancement in the interdisciplinary field of quantum computing and social network analysis. This manuscript comprehensively reviews QSNA, emphasizing its methodologies, implementation strategies, challenges, and potential applications. It explores the conceptual foundation of key social network analysis research problems, including link prediction, influence maximization, and community detection. The research examines how quantum algorithms can revolutionize such social network tasks by leveraging principles from quantum mechanics and information theory and highlights the advantages of quantum algorithms in handling complex social network structures. The implementation section delves into the practical aspects of QSNA, such as frameworks, experimental setups, and evaluation methods. We assess the capabilities of existing quantum programming language tools and platforms. Various case studies illustrate the potential of quantum computing to enhance the performance of social network analysis. Additionally, we identify several crucial challenges and future research directions for QSNA, including the complexity of developing quantum algorithms, the need for interdisciplinary knowledge, and the challenges of integrating quantum and classical computing resources. This paper aims to serve as a foundational resource for researchers and practitioners, providing insights into the transformative potential of quantum computing in advancing the analysis of social networks and outlining future research directions in this emerging field.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"117 ","pages":"Article 102808"},"PeriodicalIF":14.7,"publicationDate":"2024-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142744429","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Dual-perspective fusion for word translation enhancement 双视角融合在单词翻译中的应用

IF 14.7 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2024-11-26 DOI: 10.1016/j.inffus.2024.102815

Qiuyu Ding, Hailong Cao, Zhiqiang Cao, Tiejun Zhao

Most Bilingual Lexicon Induction (BLI) methods retrieve word translation pairs by finding the closest target word for a given source word based on cross-lingual word embeddings (WEs). However, we find that solely retrieving translation from the source-to-target perspective leads to some false positive translation pairs, which significantly harm the precision of BLI. To address this problem, we propose a novel and effective method to improve translation pair retrieval in cross-lingual WEs. Specifically, we apply a fusion of both source-side and target-side perspectives throughout the retrieval process to alleviate false positive word pairings that emanate from a single perspective. Moreover, in translation scenarios using Large Language Models (LLMs), we propose fusing the LLMs perspective with the BLI model perspective to enhance LLM’s translation capability. On benchmark datasets of BLI, our proposed method achieves competitive performance compared to existing state-of-the-art (SOTA) methods. It demonstrates effectiveness and robustness across six experimental languages, including similar language pairs and distant language pairs, under both supervised and unsupervised settings.

大多数双语词典归纳（BLI）方法基于跨语言词嵌入（WEs），通过寻找给定源词最接近的目标词来检索词翻译对。然而，我们发现，单纯从源到目标的角度检索翻译会导致一些误报翻译对，这严重损害了BLI的精度。为了解决这一问题，我们提出了一种新颖有效的方法来改进跨语言WEs中的翻译对检索。具体来说，我们在整个检索过程中应用源端和目标端视角的融合，以减轻从单一视角产生的误报词对。此外，在使用大型语言模型（LLM）的翻译场景中，我们提出将LLM视角与BLI模型视角融合，以增强LLM的翻译能力。在BLI的基准数据集上，与现有的最先进（SOTA）方法相比，我们提出的方法取得了具有竞争力的性能。它在监督和无监督设置下证明了六种实验语言的有效性和鲁棒性，包括相似的语言对和遥远的语言对。

{"title":"Dual-perspective fusion for word translation enhancement","authors":"Qiuyu Ding, Hailong Cao, Zhiqiang Cao, Tiejun Zhao","doi":"10.1016/j.inffus.2024.102815","DOIUrl":"10.1016/j.inffus.2024.102815","url":null,"abstract":"<div><div>Most Bilingual Lexicon Induction (BLI) methods retrieve word translation pairs by finding the closest target word for a given source word based on cross-lingual word embeddings (WEs). However, we find that solely retrieving translation from the source-to-target perspective leads to some false positive translation pairs, which significantly harm the precision of BLI. To address this problem, we propose a novel and effective method to improve translation pair retrieval in cross-lingual WEs. Specifically, we apply a fusion of both source-side and target-side perspectives throughout the retrieval process to alleviate false positive word pairings that emanate from a single perspective. Moreover, in translation scenarios using Large Language Models (LLMs), we propose fusing the LLMs perspective with the BLI model perspective to enhance LLM’s translation capability. On benchmark datasets of BLI, our proposed method achieves competitive performance compared to existing state-of-the-art (SOTA) methods. It demonstrates effectiveness and robustness across six experimental languages, including similar language pairs and distant language pairs, under both supervised and unsupervised settings.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"117 ","pages":"Article 102815"},"PeriodicalIF":14.7,"publicationDate":"2024-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142744311","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Security analysis and adaptive false data injection against multi-sensor fusion localization for autonomous driving 自动驾驶多传感器融合定位安全分析及自适应假数据注入

IF 14.7 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2024-11-26 DOI: 10.1016/j.inffus.2024.102822

Linqing Hu , Junqi Zhang , Jie Zhang , Shaoyin Cheng , Yuyi Wang , Weiming Zhang , Nenghai Yu

Multi-sensor Fusion (MSF) algorithms are critical components in modern autonomous driving systems, particularly in localization and AI-powered perception modules, which play a vital role in ensuring vehicle safety. The Error-State Kalman Filter (ESKF), specifically employed for localization fusion, is widely recognized for its robustness and accuracy in MSF implementations. While existing studies have demonstrated the vulnerability of ESKF to sensor spoofing attacks, these works have primarily focused on a black-box implementation, leading to an insufficient security analysis. Specifically, due to the lack of theoretical guidance in previous methods, these studies have consistently relied on exponential functions to fit attack sequences across all scenarios. As a result, the attacker had to explore an extensive parameter space to identify effective attack sequences, lacking the ability to adaptively generate optimal ones. This paper aims to fill this crucial gap by conducting a thorough security analysis of the ESKF model and presenting a simple approach for modeling injection errors in these systems. By utilizing this error modeling, we introduce a new attack strategy that employs constrained optimization to reduce the energy needed to reach the same deviation target, guaranteeing that the attack is both efficient and effective. This method increases the stealthiness of the attack, making it harder to detect. Unlike previous methods, our approach can dynamically produce nearly perfect injection signals without requiring multiple attempts to find the best parameter combination in different scenarios. Through extensive simulations and real-world experiments, we demonstrate the superiority of our method compared to state-of-the-art attack strategies. Our results indicate that our approach requires significantly less injection energy to achieve the same deviation target. Additionally, we validate the practical applicability and impact of our method through end-to-end testing on an AI-powered autonomous driving system.

多传感器融合（MSF）算法是现代自动驾驶系统的关键组成部分，特别是在定位和人工智能感知模块中，对确保车辆安全起着至关重要的作用。错误状态卡尔曼滤波器（ESKF）作为一种专门用于定位融合的算法，以其鲁棒性和准确性得到了广泛的认可。虽然现有的研究已经证明了ESKF对传感器欺骗攻击的脆弱性，但这些工作主要集中在黑盒实现上，导致安全性分析不足。具体来说，由于以前的方法缺乏理论指导，这些研究一直依赖于指数函数来拟合所有场景的攻击序列。因此，攻击者必须探索广泛的参数空间来识别有效的攻击序列，而缺乏自适应生成最优攻击序列的能力。本文旨在通过对ESKF模型进行彻底的安全性分析，并提出一种简单的方法来对这些系统中的注入错误进行建模，从而填补这一关键空白。利用这种误差建模，我们引入了一种新的攻击策略，该策略采用约束优化来减少达到相同偏差目标所需的能量，保证了攻击的高效和有效。这种方法增加了攻击的隐蔽性，使其更难被发现。与以前的方法不同，我们的方法可以动态生成近乎完美的注入信号，而无需在不同场景中多次尝试寻找最佳参数组合。通过广泛的模拟和真实世界的实验，我们证明了与最先进的攻击策略相比，我们的方法具有优越性。我们的研究结果表明，我们的方法需要更少的注入能量来达到相同的偏差目标。此外，我们还通过在人工智能驱动的自动驾驶系统上进行端到端测试，验证了我们方法的实用性和影响。

{"title":"Security analysis and adaptive false data injection against multi-sensor fusion localization for autonomous driving","authors":"Linqing Hu , Junqi Zhang , Jie Zhang , Shaoyin Cheng , Yuyi Wang , Weiming Zhang , Nenghai Yu","doi":"10.1016/j.inffus.2024.102822","DOIUrl":"10.1016/j.inffus.2024.102822","url":null,"abstract":"<div><div>Multi-sensor Fusion (MSF) algorithms are critical components in modern autonomous driving systems, particularly in localization and AI-powered perception modules, which play a vital role in ensuring vehicle safety. The Error-State Kalman Filter (ESKF), specifically employed for localization fusion, is widely recognized for its robustness and accuracy in MSF implementations. While existing studies have demonstrated the vulnerability of ESKF to sensor spoofing attacks, these works have primarily focused on a black-box implementation, leading to an insufficient security analysis. Specifically, due to the lack of theoretical guidance in previous methods, these studies have consistently relied on exponential functions to fit attack sequences across all scenarios. As a result, the attacker had to explore an extensive parameter space to identify effective attack sequences, lacking the ability to adaptively generate optimal ones. This paper aims to fill this crucial gap by conducting a thorough security analysis of the ESKF model and presenting a simple approach for modeling injection errors in these systems. By utilizing this error modeling, we introduce a new attack strategy that employs constrained optimization to reduce the energy needed to reach the same deviation target, guaranteeing that the attack is both efficient and effective. This method increases the stealthiness of the attack, making it harder to detect. Unlike previous methods, our approach can dynamically produce nearly perfect injection signals without requiring multiple attempts to find the best parameter combination in different scenarios. Through extensive simulations and real-world experiments, we demonstrate the superiority of our method compared to state-of-the-art attack strategies. Our results indicate that our approach requires significantly less injection energy to achieve the same deviation target. Additionally, we validate the practical applicability and impact of our method through end-to-end testing on an AI-powered autonomous driving system.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"117 ","pages":"Article 102822"},"PeriodicalIF":14.7,"publicationDate":"2024-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142758694","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Hybrid multivariate time series prediction system fusing transfer entropy and local relative density 融合传递熵和局部相对密度的混合多元时间序列预测系统

IF 14.7 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2024-11-26 DOI: 10.1016/j.inffus.2024.102817

Xianfeng Huang , Jianming Zhan , Weiping Ding

Kernel extreme learning machine (KELM), as a natural extension of ELM to kernel learning, has been successfully applied to solve various multivariate time series prediction (MTSP) tasks. Nevertheless, the high-dimensional and nonlinear properties of prediction information against the background of big data bring great challenges to the application of KELM. Recognizing these challenges, this paper develops a KELM-based hybrid MTSP system, aiming to address the effective mining of potential relationships among variables and sample significance. Our system is initiated by devising a feature evaluation mechanism that leverages transfer entropy and directed graph theory, effectively capturing the intricate interactions and intrinsic influences among variables. Next, we introduce a robust local relative density concept to gauge the significance level of different samples in KELM learning, and develop a more efficient KELM. Diverging from previous MTSP methodologies, the developed prediction system is capable of automatically discovering potential relationships between input features and modeling, and simultaneously realizes feature subset selection and modeling learning. Empirical evidence drawn from real-world datasets substantiates the effectiveness and practicality of our proposed system. The results not only validate our approach but also highlight its theoretical and practical superiority over existing state-of-the-art methods.

核极限学习机（KELM）作为极限学习机对核学习的自然扩展，已经成功地应用于求解各种多元时间序列预测（MTSP）任务。然而，大数据背景下预测信息的高维和非线性特性给KELM的应用带来了很大的挑战。认识到这些挑战，本文开发了一个基于kelm的混合MTSP系统，旨在有效挖掘变量和样本显著性之间的潜在关系。我们的系统是通过设计一个特征评估机制来启动的，该机制利用了传递熵和有向图理论，有效地捕获了变量之间复杂的相互作用和内在影响。接下来，我们引入了一个鲁棒的局部相对密度概念来衡量KELM学习中不同样本的显著性水平，并开发了一个更有效的KELM。与以往的MTSP方法不同，所开发的预测系统能够自动发现输入特征与建模之间的潜在关系，同时实现特征子集选择和建模学习。来自真实世界数据集的经验证据证实了我们提出的系统的有效性和实用性。结果不仅验证了我们的方法，而且突出了它在理论和实践上优于现有的最先进的方法。

{"title":"Hybrid multivariate time series prediction system fusing transfer entropy and local relative density","authors":"Xianfeng Huang , Jianming Zhan , Weiping Ding","doi":"10.1016/j.inffus.2024.102817","DOIUrl":"10.1016/j.inffus.2024.102817","url":null,"abstract":"<div><div>Kernel extreme learning machine (KELM), as a natural extension of ELM to kernel learning, has been successfully applied to solve various multivariate time series prediction (MTSP) tasks. Nevertheless, the high-dimensional and nonlinear properties of prediction information against the background of big data bring great challenges to the application of KELM. Recognizing these challenges, this paper develops a KELM-based hybrid MTSP system, aiming to address the effective mining of potential relationships among variables and sample significance. Our system is initiated by devising a feature evaluation mechanism that leverages transfer entropy and directed graph theory, effectively capturing the intricate interactions and intrinsic influences among variables. Next, we introduce a robust local relative density concept to gauge the significance level of different samples in KELM learning, and develop a more efficient KELM. Diverging from previous MTSP methodologies, the developed prediction system is capable of automatically discovering potential relationships between input features and modeling, and simultaneously realizes feature subset selection and modeling learning. Empirical evidence drawn from real-world datasets substantiates the effectiveness and practicality of our proposed system. The results not only validate our approach but also highlight its theoretical and practical superiority over existing state-of-the-art methods.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"117 ","pages":"Article 102817"},"PeriodicalIF":14.7,"publicationDate":"2024-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142744309","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Pred-ID: Future event prediction based on event type schema mining by graph induction and deduction Pred-ID：基于图归纳和演绎的事件类型模式挖掘的未来事件预测

IF 14.7 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2024-11-26 DOI: 10.1016/j.inffus.2024.102819

Huan Rong , Zhongfeng Chen , Zhenyu Lu , Xiao-ke Xu , Kai Huang , Victor S. Sheng

In the field of information management, effective event intelligence management is crucial for its development. With the continuous evolution of events, predicting future events has become a key task in information management. Event Prediction aims to predict upcoming events based on given contextual information. This requires modeling events and their relationships in the context to infer the structure of future events. However, the existing event prediction methods ignore that the event graph schema based on core events can provide more knowledge about history and future for event prediction through induction and deduction, so as to achieve accurate event prediction. In addressing this issue, we directed our focus towards Event Schema Induction. Inspired by it, we propose the Pred-ID model, designed to build event evolutionary pattern through Inductive Event Graph Generation, Deductive Event Graph Expansion, and Graph Fusion for Event Prediction. Specifically, in the Inductive Event Graph Generation phase, Pred-ID extracts the event core subgraph and event developmental trends from the instance event graph, learning the global structure and uncovering the main processes of event development. Then, in the Deductive Event Graph Expansion phase, by expanding future event node and stretching the main processes of event development into future directions, Pred-ID obtains deductive results, so as to construct the event evolutionary pattern. Finally, in the Graph Fusion for Event Prediction phase, aligning and merging the event evolutionary pattern with the instance event graph enables collaborative prediction of future events. The experimental results indicate that our proposed Pred-ID achieves optimal performance in event evolutionary pattern generation and event prediction tasks.

在信息管理领域，有效的事件智能管理对其发展至关重要。随着事件的不断演变，预测未来事件已成为信息管理中的一项关键任务。事件预测旨在根据给定的上下文信息预测即将发生的事件。这需要对事件及其在上下文中的关系进行建模，以推断未来事件的结构。然而，现有的事件预测方法忽略了基于核心事件的事件图模式可以通过归纳和演绎为事件预测提供更多关于历史和未来的知识，从而实现准确的事件预测。为了解决这个问题，我们将重点放在了事件模式归纳上。受此启发，我们提出了Pred-ID模型，旨在通过归纳事件图生成、演绎事件图展开和图融合来构建事件演化模式，用于事件预测。具体而言，在归纳事件图生成阶段，Pred-ID从实例事件图中提取事件核心子图和事件发展趋势，学习全局结构，揭示事件发展的主要过程。然后，在演绎事件图展开阶段，Pred-ID通过展开未来事件节点，将事件发展的主要过程向未来方向延伸，得到演绎结果，从而构建事件演化模式。最后，在事件预测的图融合阶段，将事件进化模式与实例事件图对齐和合并，从而实现对未来事件的协作预测。实验结果表明，本文提出的Pred-ID在事件演化模式生成和事件预测任务中取得了最优的性能。

{"title":"Pred-ID: Future event prediction based on event type schema mining by graph induction and deduction","authors":"Huan Rong , Zhongfeng Chen , Zhenyu Lu , Xiao-ke Xu , Kai Huang , Victor S. Sheng","doi":"10.1016/j.inffus.2024.102819","DOIUrl":"10.1016/j.inffus.2024.102819","url":null,"abstract":"<div><div>In the field of information management, effective event intelligence management is crucial for its development. With the continuous evolution of events, predicting future events has become a key task in information management. <em>Event Prediction</em> aims to predict upcoming events based on given contextual information. This requires modeling events and their relationships in the context to infer the structure of future events. However, the existing event prediction methods ignore that the event graph schema based on core events can provide more knowledge about history and future for event prediction through induction and deduction, so as to achieve accurate event prediction. In addressing this issue, we directed our focus towards <em>Event Schema Induction</em>. Inspired by it, we propose the <strong><em>Pred-ID</em></strong> model, designed to build event evolutionary pattern through <em>Inductive Event Graph Generation</em>, <em>Deductive Event Graph Expansion</em>, and <em>Graph Fusion for Event Prediction</em>. Specifically, in the <em>Inductive Event Graph Generation</em> phase, Pred-ID extracts the event core subgraph and event developmental trends from the instance event graph, learning the global structure and uncovering the main processes of event development. Then, in the <em>Deductive Event Graph Expansion</em> phase, by expanding future event node and stretching the main processes of event development into future directions, Pred-ID obtains deductive results, so as to construct the event evolutionary pattern. Finally, in the <em>Graph Fusion for Event Prediction</em> phase, aligning and merging the event evolutionary pattern with the instance event graph enables collaborative prediction of future events. The experimental results indicate that our proposed Pred-ID achieves optimal performance in event evolutionary pattern generation and event prediction tasks.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"117 ","pages":"Article 102819"},"PeriodicalIF":14.7,"publicationDate":"2024-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142744431","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

FedKD-IDS: A robust intrusion detection system using knowledge distillation-based semi-supervised federated learning and anti-poisoning attack mechanism FedKD-IDS：基于知识提炼的半监督联合学习和反中毒攻击机制的稳健入侵检测系统

IF 14.7 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2024-11-26 DOI: 10.1016/j.inffus.2024.102807

Nguyen Huu Quyen, Phan The Duy, Ngo Thao Nguyen, Nghi Hoang Khoa, Van-Hau Pham

In the realm of the Internet of Things (IoT), there has been a notable increase in the development and efficacy of Intrusion Detection Systems (IDS) that leverage machine learning (ML). Specifically, Federated Learning-based IDSs (FL-based IDS) have witnessed significant growth. These systems aim to mitigate data privacy breaches and minimize the communication overhead associated with dataset collection. Limited hardware resources also pose a significant constraint, preventing numerous IoT devices from actively engaging in FL. However, despite these advancements, certain challenges persist in the research domain. Issues such as elevated communication overhead, the potential for recovering private data, non-independent and identically distributed (Non-IID) data and a scarcity of labeled data remain noteworthy concerns. Additionally, vulnerabilities exist in the server-client communication during the FL process, creating opportunities for attackers to execute poisoning attacks on the client side with relative ease. To address these challenges, our paper introduces a semi-supervised approach for FL-based IDS. Our approach, named FedKD-IDS, employs knowledge distillation with a voting mechanism in place of weighted parameter aggregation and incorporates an anti-poisoning method. We conducted experiments to evaluate the effectiveness of our approach across diverse scenarios, including scenarios with Non-IID and varying data distributions. Additionally, we investigated various rates of malicious collaboration to demonstrate their impact in the federated training process. The results obtained from the real-world N-BaIoT dataset indicate that our approach surpasses the performance of the state-of-the-art (SOTA) SSFL method. Especially, even in the context of a poisoning attack where 50% of all collaborators targeted label flipping attack, FedKD-IDS demonstrated an accuracy of 79%, surpassing SSFL, which achieved only 19.86%. Furthermore, the outcomes also validated that the FedKD-IDS method has the capability to exclude over 85% of malicious collaborators during the aggregation phase of the federated training process.

在物联网（IoT）领域，利用机器学习（ML）的入侵检测系统（IDS）的开发和功效显著提高。具体来说，基于联合学习的入侵检测系统（FL-based IDS）得到了显著发展。这些系统旨在减少数据隐私泄露，并最大限度地降低与数据集收集相关的通信开销。有限的硬件资源也是一个重要的制约因素，使众多物联网设备无法主动参与 FL。然而，尽管取得了这些进展，研究领域仍存在某些挑战。通信开销增大、恢复私人数据的可能性、非独立和同分布（Non-IID）数据以及标记数据稀缺等问题仍然值得关注。此外，在 FL 过程中，服务器与客户端的通信存在漏洞，这为攻击者在客户端轻松实施中毒攻击创造了机会。为了应对这些挑战，我们的论文为基于 FL 的 IDS 引入了一种半监督方法。我们的方法被命名为 FedKD-IDS，它采用知识提炼和投票机制来代替加权参数聚合，并结合了一种反中毒方法。我们进行了实验，以评估我们的方法在不同场景下的有效性，包括非 IID 场景和不同的数据分布。此外，我们还研究了各种恶意协作率，以证明它们在联合训练过程中的影响。从真实世界 N-BaIoT 数据集获得的结果表明，我们的方法超越了最先进（SOTA）SSFL 方法的性能。特别是，即使在中毒攻击的情况下，50% 的合作者针对标签翻转攻击，FedKD-IDS 的准确率也达到了 79%，超过了仅为 19.86% 的 SSFL。此外，研究结果还验证了 FedKD-IDS 方法有能力在联合训练过程的聚合阶段排除 85% 以上的恶意合作者。

{"title":"FedKD-IDS: A robust intrusion detection system using knowledge distillation-based semi-supervised federated learning and anti-poisoning attack mechanism","authors":"Nguyen Huu Quyen, Phan The Duy, Ngo Thao Nguyen, Nghi Hoang Khoa, Van-Hau Pham","doi":"10.1016/j.inffus.2024.102807","DOIUrl":"10.1016/j.inffus.2024.102807","url":null,"abstract":"<div><div>In the realm of the Internet of Things (IoT), there has been a notable increase in the development and efficacy of Intrusion Detection Systems (IDS) that leverage machine learning (ML). Specifically, Federated Learning-based IDSs (FL-based IDS) have witnessed significant growth. These systems aim to mitigate data privacy breaches and minimize the communication overhead associated with dataset collection. Limited hardware resources also pose a significant constraint, preventing numerous IoT devices from actively engaging in FL. However, despite these advancements, certain challenges persist in the research domain. Issues such as elevated communication overhead, the potential for recovering private data, non-independent and identically distributed (Non-IID) data and a scarcity of labeled data remain noteworthy concerns. Additionally, vulnerabilities exist in the server-client communication during the FL process, creating opportunities for attackers to execute poisoning attacks on the client side with relative ease. To address these challenges, our paper introduces a semi-supervised approach for FL-based IDS. Our approach, named FedKD-IDS, employs knowledge distillation with a voting mechanism in place of weighted parameter aggregation and incorporates an anti-poisoning method. We conducted experiments to evaluate the effectiveness of our approach across diverse scenarios, including scenarios with Non-IID and varying data distributions. Additionally, we investigated various rates of malicious collaboration to demonstrate their impact in the federated training process. The results obtained from the real-world N-BaIoT dataset indicate that our approach surpasses the performance of the state-of-the-art (SOTA) SSFL method. Especially, even in the context of a poisoning attack where 50% of all collaborators targeted label flipping attack, FedKD-IDS demonstrated an accuracy of 79%, surpassing SSFL, which achieved only 19.86%. Furthermore, the outcomes also validated that the FedKD-IDS method has the capability to exclude over 85% of malicious collaborators during the aggregation phase of the federated training process.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"117 ","pages":"Article 102807"},"PeriodicalIF":14.7,"publicationDate":"2024-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142721317","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Towards marine snow removal with fusing Fourier information 基于傅里叶信息融合的海洋除雪方法研究

IF 14.7 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2024-11-26 DOI: 10.1016/j.inffus.2024.102810

Yakun Ju , Jun Xiao , Cong Zhang , Hao Xie , Anwei Luo , Huiyu Zhou , Junyu Dong , Alex C. Kot

Marine snow, caused by the aggregation of small organic and inorganic particles, creates a visual effect similar to drifting snowflakes. Traditional methods for removing marine snow often use median filtering, which can blur the entire image. Although deep learning approaches attempt to address this issue, they typically only work in the spatial domain and still struggle with blurring and residual marine snow artifacts. These challenges arise because the spatial domain alone cannot easily distinguish between real object structures and noise-like marine snow artifacts. To address this, we propose the Deep Fourier Marine Snow Removal Network (DF-MSRN), which integrates both spatial and Fourier domain information to effectively restore images affected by marine snow. DF-MSRN employs a two-stage approach that leverages both Fourier frequency and spatial information: it first estimates a restored map of the amplitude component to address particle removal, avoiding additional noise in the spatial domain. Then, a fusion module combines Fourier frequency global information with spatial local information to refine image details. Experimental results show that DF-MSRN significantly outperforms existing denoising techniques on various marine image datasets, enhancing image clarity and detail preservation.

海洋雪是由微小的有机和无机颗粒聚集而成的，它的视觉效果类似于飘雪。传统的去除海洋积雪的方法通常采用中值滤波，这可能会模糊整个图像。尽管深度学习方法试图解决这个问题，但它们通常只在空间领域起作用，并且仍然在与模糊和残留的海洋雪伪影作斗争。这些挑战的出现是因为单独的空间域无法轻易区分真实的物体结构和类似噪声的海洋雪人工制品。为了解决这一问题，我们提出了深度傅里叶海洋除雪网络（DF-MSRN），该网络集成了空间和傅里叶域信息，可以有效地恢复受海洋雪影响的图像。DF-MSRN采用两阶段方法，利用傅里叶频率和空间信息：它首先估计振幅分量的恢复图，以解决粒子去除问题，避免空间域中的额外噪声。然后，融合模块将傅里叶频率全局信息与空间局部信息相结合，对图像细节进行细化；实验结果表明，DF-MSRN在各种海洋图像数据集上显著优于现有的去噪技术，增强了图像的清晰度和细节保留。

{"title":"Towards marine snow removal with fusing Fourier information","authors":"Yakun Ju , Jun Xiao , Cong Zhang , Hao Xie , Anwei Luo , Huiyu Zhou , Junyu Dong , Alex C. Kot","doi":"10.1016/j.inffus.2024.102810","DOIUrl":"10.1016/j.inffus.2024.102810","url":null,"abstract":"<div><div>Marine snow, caused by the aggregation of small organic and inorganic particles, creates a visual effect similar to drifting snowflakes. Traditional methods for removing marine snow often use median filtering, which can blur the entire image. Although deep learning approaches attempt to address this issue, they typically only work in the spatial domain and still struggle with blurring and residual marine snow artifacts. These challenges arise because the spatial domain alone cannot easily distinguish between real object structures and noise-like marine snow artifacts. To address this, we propose the Deep Fourier Marine Snow Removal Network (DF-MSRN), which integrates both spatial and Fourier domain information to effectively restore images affected by marine snow. DF-MSRN employs a two-stage approach that leverages both Fourier frequency and spatial information: it first estimates a restored map of the amplitude component to address particle removal, avoiding additional noise in the spatial domain. Then, a fusion module combines Fourier frequency global information with spatial local information to refine image details. Experimental results show that DF-MSRN significantly outperforms existing denoising techniques on various marine image datasets, enhancing image clarity and detail preservation.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"117 ","pages":"Article 102810"},"PeriodicalIF":14.7,"publicationDate":"2024-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142744308","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A temporally insensitive spatio-temporal fusion method for remote sensing imagery via semantic prior regularization 基于语义先验正则化的遥感影像时空不敏感融合方法

IF 14.7 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2024-11-24 DOI: 10.1016/j.inffus.2024.102818

Qiang Liu , Xiangchao Meng , Shenfu Zhang , Xuebin Li , Feng Shao

Spatio-temporal fusion has become a popular technology for generating remote sensing images with high spatial and high temporal resolutions, thus providing valuable data support for remote sensing monitoring applications, such as environmental monitoring and city planning. Currently, deep learning-based methods have garnered a significant amount of attention, and they mostly employ the fine image at the neighboring date as an auxiliary image. However, capturing usable neighboring fine images may be challenging due to the adverse effects of weather conditions on optical images. Moreover, the fusion performance drops sharply when the temporal interval is long (i.e., there are significant differences in images). In this paper, we proposed a bidirectional pyramid fusion network with semantic prior regularization (BPFN-SPR), which exhibits remarkable flexibility and robustness to temporal intervals.

Specifically, the proposed BPFN-SPR contains dual-path operations (i.e., Semantic Extraction path and Image Reconstruction path). The semantic extraction path has two modes: parameter learning mode and parameter freezing mode. The parameter learning mode aims to learn the information representation of the auxiliary fine image, while the parameter freezing mode aims to perceive the accurate semantic information of the target fine image. The image reconstruction path progressively reconstructs spatial details of fine images from coarse images, which jointly optimizes the target fine image and the auxiliary fine image, reducing the temporal sensitivity of the reconstruction branch, and thereby improving its generalization ability. Experimental results show that the proposed method has competitive performance, especially for areas with land cover changes. In addition, extensive experiments using images at multi-temporal intervals as auxiliary images have also demonstrated the significant advantages of the proposed method. The mean PSNR value attains 31.0713, while the average spectral index SAM measures 0.1640 on the LGC test set. Meanwhile, for the CIA test set, the average PSNR is recorded at 29.5332, accompanied by an average spectral index SAM of 0.1865. Therefore, the proposed BPFN-SPR has considerable potential in monitoring Earth's surface dynamics.

时空融合技术已成为高时空分辨率遥感图像生成的热门技术，为环境监测、城市规划等遥感监测应用提供了宝贵的数据支持。目前，基于深度学习的方法已经获得了大量的关注，它们大多使用邻近日期的精细图像作为辅助图像。然而，由于天气条件对光学图像的不利影响，捕获可用的邻近精细图像可能具有挑战性。当时间间隔较长（即图像之间存在显著差异）时，融合性能急剧下降。本文提出了一种具有语义先验正则化的双向金字塔融合网络（BPFN-SPR），该网络对时间区间具有良好的灵活性和鲁棒性。具体来说，所提出的BPFN-SPR包含双路径操作（即语义提取路径和图像重建路径）。语义提取路径有参数学习模式和参数冻结模式两种模式。参数学习模式的目的是学习辅助精细图像的信息表示，参数冻结模式的目的是感知目标精细图像的准确语义信息。图像重建路径从粗图像逐步重建精细图像的空间细节，共同优化目标精细图像和辅助精细图像，降低重建分支的时间灵敏度，从而提高其泛化能力。实验结果表明，该方法具有较好的性能，特别是在土地覆被变化较大的地区。此外，利用多时间间隔图像作为辅助图像的大量实验也证明了该方法的显著优势。在LGC测试集上，平均PSNR值为31.0713，平均光谱指数SAM值为0.1640。同时，CIA测试集的平均PSNR为29.5332，平均光谱指数SAM为0.1865。因此，所提出的BPFN-SPR在监测地球表面动力学方面具有相当大的潜力。

{"title":"A temporally insensitive spatio-temporal fusion method for remote sensing imagery via semantic prior regularization","authors":"Qiang Liu , Xiangchao Meng , Shenfu Zhang , Xuebin Li , Feng Shao","doi":"10.1016/j.inffus.2024.102818","DOIUrl":"10.1016/j.inffus.2024.102818","url":null,"abstract":"<div><div>Spatio-temporal fusion has become a popular technology for generating remote sensing images with high spatial and high temporal resolutions, thus providing valuable data support for remote sensing monitoring applications, such as environmental monitoring and city planning. Currently, deep learning-based methods have garnered a significant amount of attention, and they mostly employ the fine image at the neighboring date as an auxiliary image. However, capturing usable neighboring fine images may be challenging due to the adverse effects of weather conditions on optical images. Moreover, the fusion performance drops sharply when the temporal interval is long (i.e., there are significant differences in images). In this paper, we proposed a bidirectional pyramid fusion network with semantic prior regularization (BPFN-SPR), which exhibits remarkable flexibility and robustness to temporal intervals.</div><div>Specifically, the proposed BPFN-SPR contains dual-path operations (i.e., Semantic Extraction path and Image Reconstruction path). The semantic extraction path has two modes: parameter learning mode and parameter freezing mode. The parameter learning mode aims to learn the information representation of the auxiliary fine image, while the parameter freezing mode aims to perceive the accurate semantic information of the target fine image. The image reconstruction path progressively reconstructs spatial details of fine images from coarse images, which jointly optimizes the target fine image and the auxiliary fine image, reducing the temporal sensitivity of the reconstruction branch, and thereby improving its generalization ability. Experimental results show that the proposed method has competitive performance, especially for areas with land cover changes. In addition, extensive experiments using images at multi-temporal intervals as auxiliary images have also demonstrated the significant advantages of the proposed method. The mean PSNR value attains 31.0713, while the average spectral index SAM measures 0.1640 on the LGC test set. Meanwhile, for the CIA test set, the average PSNR is recorded at 29.5332, accompanied by an average spectral index SAM of 0.1865. Therefore, the proposed BPFN-SPR has considerable potential in monitoring Earth's surface dynamics.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"117 ","pages":"Article 102818"},"PeriodicalIF":14.7,"publicationDate":"2024-11-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142759032","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

IF-USOD: Multimodal information fusion interactive feature enhancement architecture for underwater salient object detection IF-USOD：用于水下突出物体探测的多模态信息融合交互式特征增强架构

IF 14.7 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2024-11-23 DOI: 10.1016/j.inffus.2024.102806

Genji Yuan , Jintao Song , Jinjiang Li

Underwater salient object detection (USOD) has garnered increasing attention due to its superior performance in various underwater visual tasks. Despite the growing interest, research on USOD remains in its nascent stages, with existing methods often struggling to capture long-range contextual features of salient objects. Additionally, these methods frequently overlook the complementary nature of multimodal information. The multimodal information fusion can render previously indiscernible objects more detectable, as capturing complementary features from diverse source images enables a more accurate depiction of objects. In this work, we explore an innovative approach that integrates RGB and depth information, coupled with interactive feature enhancement, to advance the detection of underwater salient objects. Our method first leverages the strengths of both transformer and convolutional neural network architectures to extract features from source images. Here, we employ a two-stage training strategy designed to optimize feature fusion. Subsequently, we utilize self-attention and cross-attention mechanisms to model the correlations among the extracted features, thereby amplifying the relevant features. Finally, to fully exploit features across different network layers, we introduce a cross-scale learning strategy to facilitate multi-scale feature fusion, which improves the detection accuracy of underwater salient objects by generating both coarse and fine salient predictions. Extensive experimental evaluations demonstrate the state-of-the-art model performance of our proposed method.

水下突出物体检测（USOD）因其在各种水下视觉任务中的卓越表现而受到越来越多的关注。尽管人们的兴趣与日俱增，但有关水下突出物体检测的研究仍处于起步阶段，现有方法往往难以捕捉突出物体的长距离上下文特征。此外，这些方法经常忽略多模态信息的互补性。多模态信息融合可以使以前无法识别的物体更容易被检测到，因为从不同的源图像中捕捉互补特征可以更准确地描述物体。在这项工作中，我们探索了一种创新方法，将 RGB 和深度信息与交互式特征增强相结合，以推进水下突出物体的检测。我们的方法首先利用变压器和卷积神经网络架构的优势，从源图像中提取特征。在此，我们采用了两阶段训练策略，旨在优化特征融合。随后，我们利用自注意和交叉注意机制对提取的特征之间的相关性进行建模，从而放大相关特征。最后，为了充分利用不同网络层的特征，我们引入了跨尺度学习策略，以促进多尺度特征融合，通过生成粗略和精细的突出预测，提高水下突出物体的检测精度。广泛的实验评估证明了我们提出的方法具有最先进的模型性能。

{"title":"IF-USOD: Multimodal information fusion interactive feature enhancement architecture for underwater salient object detection","authors":"Genji Yuan , Jintao Song , Jinjiang Li","doi":"10.1016/j.inffus.2024.102806","DOIUrl":"10.1016/j.inffus.2024.102806","url":null,"abstract":"<div><div>Underwater salient object detection (USOD) has garnered increasing attention due to its superior performance in various underwater visual tasks. Despite the growing interest, research on USOD remains in its nascent stages, with existing methods often struggling to capture long-range contextual features of salient objects. Additionally, these methods frequently overlook the complementary nature of multimodal information. The multimodal information fusion can render previously indiscernible objects more detectable, as capturing complementary features from diverse source images enables a more accurate depiction of objects. In this work, we explore an innovative approach that integrates RGB and depth information, coupled with interactive feature enhancement, to advance the detection of underwater salient objects. Our method first leverages the strengths of both transformer and convolutional neural network architectures to extract features from source images. Here, we employ a two-stage training strategy designed to optimize feature fusion. Subsequently, we utilize self-attention and cross-attention mechanisms to model the correlations among the extracted features, thereby amplifying the relevant features. Finally, to fully exploit features across different network layers, we introduce a cross-scale learning strategy to facilitate multi-scale feature fusion, which improves the detection accuracy of underwater salient objects by generating both coarse and fine salient predictions. Extensive experimental evaluations demonstrate the state-of-the-art model performance of our proposed method.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"117 ","pages":"Article 102806"},"PeriodicalIF":14.7,"publicationDate":"2024-11-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142721319","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Physical prior-guided deep fusion network with shading cues for shape from polarization 物理先验引导的深度融合网络，通过偏振的阴影线索了解形状

IF 14.7 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2024-11-23 DOI: 10.1016/j.inffus.2024.102805

Rui Liu , Zhiyuan Zhang , Yini Peng , Jiayi Ma , Xin Tian

Shape from polarization (SfP) is a powerful passive three-dimensional imaging technique that enables the reconstruction of surface normal with dense textural details. However, existing deep learning-based SfP methods only focus on the polarization prior, which makes it difficult to accurately reconstruct targets with rich texture details under complicated scenes. Aiming to improve the reconstruction accuracy, we utilize the surface normal estimated from shading cues and the innovatively proposed specular confidence as shading prior to provide additional feature information. Furthermore, to efficiently combine the polarization and shading priors, a novel deep fusion network named SfPSNet is proposed for the information extraction and the reconstruction of surface normal. SfPSNet is implemented based on a dual-branch architecture to handle different physical priors. A feature correction module is specifically designed to mutually rectify the defects in channel-wise and spatial-wise dimensions, respectively. In addition, a feature fusion module is proposed to fuse the feature maps of polarization and shading priors based on an efficient cross-attention mechanism. Our experimental results show that the fusion of polarization and shading priors can significantly improve the reconstruction quality of surface normal, especially for objects or scenes illuminated by complex lighting sources. As a result, SfPSNet shows state-of-the-art performance compared with existing deep learning-based SfP methods benefiting from its efficiency in extracting and fusing information from different priors.

来自偏振的形状（SfP）是一种强大的被动三维成像技术，能够重建具有密集纹理细节的表面法线。然而，现有的基于深度学习的 SfP 方法只关注偏振先验，难以在复杂场景下准确重建具有丰富纹理细节的目标。为了提高重建精度，我们利用阴影线索估算的表面法线和创新性地提出的镜面置信度作为阴影先验，以提供额外的特征信息。此外，为了有效结合偏振先验和阴影先验，我们提出了一种名为 SfPSNet 的新型深度融合网络，用于信息提取和表面法线重建。SfPSNet 基于双分支架构实现，可处理不同的物理前验。专门设计了一个特征校正模块，分别在通道维度和空间维度上相互修正缺陷。此外，我们还提出了一个特征融合模块，基于高效的交叉注意机制融合偏振和阴影先验的特征图。实验结果表明，偏振和阴影前验的融合能显著提高表面法线的重建质量，尤其是对于复杂光源照射的物体或场景。因此，与现有的基于深度学习的 SfP 方法相比，SfPSNet 得益于其从不同前验中提取和融合信息的效率，表现出了最先进的性能。

{"title":"Physical prior-guided deep fusion network with shading cues for shape from polarization","authors":"Rui Liu , Zhiyuan Zhang , Yini Peng , Jiayi Ma , Xin Tian","doi":"10.1016/j.inffus.2024.102805","DOIUrl":"10.1016/j.inffus.2024.102805","url":null,"abstract":"<div><div>Shape from polarization (SfP) is a powerful passive three-dimensional imaging technique that enables the reconstruction of surface normal with dense textural details. However, existing deep learning-based SfP methods only focus on the polarization prior, which makes it difficult to accurately reconstruct targets with rich texture details under complicated scenes. Aiming to improve the reconstruction accuracy, we utilize the surface normal estimated from shading cues and the innovatively proposed specular confidence as shading prior to provide additional feature information. Furthermore, to efficiently combine the polarization and shading priors, a novel deep fusion network named SfPSNet is proposed for the information extraction and the reconstruction of surface normal. SfPSNet is implemented based on a dual-branch architecture to handle different physical priors. A feature correction module is specifically designed to mutually rectify the defects in channel-wise and spatial-wise dimensions, respectively. In addition, a feature fusion module is proposed to fuse the feature maps of polarization and shading priors based on an efficient cross-attention mechanism. Our experimental results show that the fusion of polarization and shading priors can significantly improve the reconstruction quality of surface normal, especially for objects or scenes illuminated by complex lighting sources. As a result, SfPSNet shows state-of-the-art performance compared with existing deep learning-based SfP methods benefiting from its efficiency in extracting and fusing information from different priors.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"117 ","pages":"Article 102805"},"PeriodicalIF":14.7,"publicationDate":"2024-11-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142721314","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0