Information Fusion最新文献_第7页

A data fusion approach to synthesize microwave imagery of tropical cyclones from infrared data using vision transformers 利用视觉变换从红外数据合成热带气旋微波图像的数据融合方法

IF 15.5 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2026-01-22 DOI: 10.1016/j.inffus.2026.104167

Fan Meng , Tao Song , Xianxuan Lin , Kunlin Yang

Microwave images with high spatiotemporal resolution are essential for observing and predicting tropical cyclones (TCs), including TC positioning, intensity estimation, and detection of concentric eyewall. Nevertheless, the temporal resolution of tropical cyclone microwave (TCMW) images is limited due to satellite quantity and orbit constraints, presenting a challenging problem for TC disaster forecasting. This research suggests a multi-sensor data fusion approach, using high-temporal-resolution tropical cyclone infrared (TCIR) images to generate synthetic TCMW images, offering a solution to this data scarcity problem. In particular, we introduce a deep learning network based on the Vision Transformer (TCA-ViT) to translate TCIR images into TCMW images. This can be viewed as a form of synthetic data generation, enhancing the available information for decision-making. We integrate a phase-based physical guidance mechanism into the training process. Furthermore, we have developed a dataset of TC infrared-to-microwave image conversions (TCIR2MW) for training and testing the model. Experimental results demonstrate the method’s capability in rapidly and accurately extracting key features of TCs. Leveraging techniques like Mask and Transfer Learning, it addresses the absence of TCMW images by generating MW images from IR images, thereby aiding downstream tasks like TC intensity and precipitation forecasting. This study introduces a novel approach to the field of TC image research, with the potential to advance deep learning in this direction and provide vital insights for real-time observation and prediction of global TCs. Our source code and data are publicly available online at https://github.com/kleenY/TCIR2MW.

具有高时空分辨率的微波图像对热带气旋的观测和预报至关重要，包括热带气旋定位、强度估计和同心眼壁检测。然而，由于卫星数量和轨道的限制，热带气旋微波图像的时间分辨率有限，这给热带气旋灾害预报带来了挑战。本研究提出了一种多传感器数据融合方法，利用高时间分辨率热带气旋红外（TCIR）图像生成合成的热带气旋红外（TCMW）图像，解决了这一数据稀缺问题。特别地，我们引入了一种基于视觉转换器（TCA-ViT）的深度学习网络，将TCIR图像转换为TCMW图像。这可以看作是生成综合数据的一种形式，增强了决策所需的现有信息。我们将基于阶段的物理指导机制整合到训练过程中。此外，我们开发了一个TC红外到微波图像转换（TCIR2MW）数据集，用于训练和测试模型。实验结果表明，该方法能够快速准确地提取tc的关键特征。利用掩模和迁移学习等技术，它通过从红外图像生成MW图像来解决TCMW图像的缺失问题，从而帮助下游任务，如TC强度和降水预报。本研究为TC图像研究领域引入了一种新的方法，有可能在这一方向上推进深度学习，并为全球TC的实时观测和预测提供重要的见解。我们的源代码和数据可以在https://github.com/kleenY/TCIR2MW上公开获取。

{"title":"A data fusion approach to synthesize microwave imagery of tropical cyclones from infrared data using vision transformers","authors":"Fan Meng , Tao Song , Xianxuan Lin , Kunlin Yang","doi":"10.1016/j.inffus.2026.104167","DOIUrl":"10.1016/j.inffus.2026.104167","url":null,"abstract":"<div><div>Microwave images with high spatiotemporal resolution are essential for observing and predicting tropical cyclones (TCs), including TC positioning, intensity estimation, and detection of concentric eyewall. Nevertheless, the temporal resolution of tropical cyclone microwave (TCMW) images is limited due to satellite quantity and orbit constraints, presenting a challenging problem for TC disaster forecasting. This research suggests a multi-sensor data fusion approach, using high-temporal-resolution tropical cyclone infrared (TCIR) images to generate synthetic TCMW images, offering a solution to this data scarcity problem. In particular, we introduce a deep learning network based on the Vision Transformer (TCA-ViT) to translate TCIR images into TCMW images. This can be viewed as a form of synthetic data generation, enhancing the available information for decision-making. We integrate a phase-based physical guidance mechanism into the training process. Furthermore, we have developed a dataset of TC infrared-to-microwave image conversions (TCIR2MW) for training and testing the model. Experimental results demonstrate the method’s capability in rapidly and accurately extracting key features of TCs. Leveraging techniques like Mask and Transfer Learning, it addresses the absence of TCMW images by generating MW images from IR images, thereby aiding downstream tasks like TC intensity and precipitation forecasting. This study introduces a novel approach to the field of TC image research, with the potential to advance deep learning in this direction and provide vital insights for real-time observation and prediction of global TCs. Our source code and data are publicly available online at <span><span>https://github.com/kleenY/TCIR2MW</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"131 ","pages":"Article 104167"},"PeriodicalIF":15.5,"publicationDate":"2026-01-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146033286","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

ELOGOnet: Knowledge-enhanced local-global learning for cardiac diagnosis ELOGOnet：用于心脏诊断的知识增强局部-全局学习

IF 15.5 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2026-01-21 DOI: 10.1016/j.inffus.2026.104168

Yizhuo Feng , Beibei Wang , Zirui Wang , Ke Jiang , Peng Wang , Lidong Du , Xianxiang Chen , Pang Wu , Zhenfeng Li , Junxian Song , Libin Jiang , Zhen Fang

The diagnostic process of a human cardiologist is a holistic act of reasoning that seamlessly integrates two key components: (1) a synergistic analysis of the ECG signal itself, combining insights from both global rhythmic patterns and local morphologies; and (2) a prior-informed interpretation process that leverages internalized medical priors and external patient-specific information. However, existing deep learning models struggle to emulate this complex expert reasoning, often facing a dual dilemma: a failure to synergize local and global features within a unified framework, and a widespread neglect of valuable, low-cost prior knowledge sources like disease associations and patient metadata. To bridge this gap, we propose ELOGOnet, a novel deep learning framework designed to model the expert diagnostic workflow. Modeling the expert’s synergistic signal analysis, ELOGOnet employs a parallel hybrid architecture that integrates a State Space Model (SSM) for global rhythms and a CNN for local morphologies. Enabling a prior-informed interpretation, the framework incorporates two key innovations: an association loss that enhances clinical coherence by modeling disease comorbidity and mutual exclusivity, and an adaptive cross-gating module for the robust fusion of patient metadata. Extensive experiments on several mainstream public benchmarks demonstrate that ELOGOnet establishes a new state-of-the-art by achieving an average Macro-F1 of 63.8% across 8 multi-label tasks and consistently outperforming 16 competitive baselines, thereby setting a new performance benchmark for automated cardiac diagnosis from ECG.

人类心脏病专家的诊断过程是一个整体的推理行为，它无缝集成了两个关键组成部分：(1)对ECG信号本身的协同分析，结合来自全局节律模式和局部形态的见解；(2)利用内化医疗经验和外部患者特定信息的先验信息解释过程。然而，现有的深度学习模型很难模仿这种复杂的专家推理，往往面临双重困境：无法在统一的框架内协同局部和全局特征，以及普遍忽视有价值的、低成本的先验知识来源，如疾病关联和患者元数据。为了弥补这一差距，我们提出了ELOGOnet，这是一个新的深度学习框架，旨在对专家诊断工作流程进行建模。ELOGOnet采用并行混合架构，为专家的协同信号分析建模，该架构集成了用于全局节奏的状态空间模型（SSM）和用于局部形态的CNN。该框架采用了两个关键的创新：通过建模疾病共病和互斥性来增强临床一致性的关联丢失，以及用于稳健融合患者元数据的自适应交叉门控模块。在几个主流公共基准上进行的大量实验表明，ELOGOnet通过在8个多标签任务中实现平均63.8%的Macro-F1，并始终优于16个竞争基准，从而建立了新的性能基准，从而为心电图自动心脏诊断设定了新的性能基准。

{"title":"ELOGOnet: Knowledge-enhanced local-global learning for cardiac diagnosis","authors":"Yizhuo Feng , Beibei Wang , Zirui Wang , Ke Jiang , Peng Wang , Lidong Du , Xianxiang Chen , Pang Wu , Zhenfeng Li , Junxian Song , Libin Jiang , Zhen Fang","doi":"10.1016/j.inffus.2026.104168","DOIUrl":"10.1016/j.inffus.2026.104168","url":null,"abstract":"<div><div>The diagnostic process of a human cardiologist is a holistic act of reasoning that seamlessly integrates two key components: (1) a synergistic analysis of the ECG signal itself, combining insights from both global rhythmic patterns and local morphologies; and (2) a prior-informed interpretation process that leverages internalized medical priors and external patient-specific information. However, existing deep learning models struggle to emulate this complex expert reasoning, often facing a dual dilemma: a failure to synergize local and global features within a unified framework, and a widespread neglect of valuable, low-cost prior knowledge sources like disease associations and patient metadata. To bridge this gap, we propose ELOGOnet, a novel deep learning framework designed to model the expert diagnostic workflow. Modeling the expert’s synergistic signal analysis, ELOGOnet employs a parallel hybrid architecture that integrates a State Space Model (SSM) for global rhythms and a CNN for local morphologies. Enabling a prior-informed interpretation, the framework incorporates two key innovations: an association loss that enhances clinical coherence by modeling disease comorbidity and mutual exclusivity, and an adaptive cross-gating module for the robust fusion of patient metadata. Extensive experiments on several mainstream public benchmarks demonstrate that ELOGOnet establishes a new state-of-the-art by achieving an average Macro-F1 of 63.8% across 8 multi-label tasks and consistently outperforming 16 competitive baselines, thereby setting a new performance benchmark for automated cardiac diagnosis from ECG.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"131 ","pages":"Article 104168"},"PeriodicalIF":15.5,"publicationDate":"2026-01-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146033293","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Multimodal fusion of 3D point cloud and intraoperative imaging to enhance surgical robot navigation 三维点云和术中影像的多模态融合增强手术机器人导航

IF 15.5 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2026-01-21 DOI: 10.1016/j.inffus.2026.104171

Yiheng Wang , Tianlun Wang , Tao Liu , Yihao Huang

To address the insufficient navigation accuracy of surgical robots in dynamic, complex, and non-rigid intraoperative environments, this paper proposes an enhanced multimodal fusion framework—EMF-RSN. This framework achieves spatial consistency between point clouds and intraoperative images through a depth-guided Geometry-Vision Alignment Module (GVAN), implements dynamic weighted fusion of geometric and visual features through a cross-modal attention fusion module (CAFM), and constructs a closed-loop optimization mechanism from perception to decision through a Task Feedback Optimization (TFO) module, thereby improving navigation accuracy and stability.

Experiments on the public dataset (Hamlyn) and the self-built simulation dataset (Sim-Surgical Fusion) demonstrate that EMF-RSN significantly outperforms existing methods in terms of geometric accuracy, semantic consistency, and task robustness. Compared to traditional registration algorithms, point cloud errors are reduced by approximately 50%, trajectory errors are reduced by over 20%, and real-time performance of 44 FPS is maintained even in complex deformation and occlusion environments. This research provides a new technical approach and model foundation for realizing intelligent surgical navigation that integrates virtual and real elements, and is of great significance for the perception and autonomous control of surgical robots.

为了解决手术机器人在动态、复杂、非刚性的术中环境中导航精度不足的问题，本文提出了一种增强型多模态融合框架- emf - rsn。该框架通过深度引导的几何-视觉对齐模块（GVAN）实现点云和术中图像的空间一致性，通过跨模态注意力融合模块（CAFM）实现几何特征和视觉特征的动态加权融合，通过任务反馈优化模块（TFO）构建从感知到决策的闭环优化机制，从而提高导航精度和稳定性。在公共数据集（Hamlyn）和自建仿真数据集（Sim-Surgical Fusion）上的实验表明，EMF-RSN在几何精度、语义一致性和任务鲁棒性方面显著优于现有方法。与传统配准算法相比，点云误差减少了约50%，轨迹误差减少了20%以上，即使在复杂的变形和遮挡环境下也能保持44 FPS的实时性能。本研究为实现虚实结合的智能手术导航提供了新的技术途径和模型基础，对手术机器人的感知和自主控制具有重要意义。

{"title":"Multimodal fusion of 3D point cloud and intraoperative imaging to enhance surgical robot navigation","authors":"Yiheng Wang , Tianlun Wang , Tao Liu , Yihao Huang","doi":"10.1016/j.inffus.2026.104171","DOIUrl":"10.1016/j.inffus.2026.104171","url":null,"abstract":"<div><div>To address the insufficient navigation accuracy of surgical robots in dynamic, complex, and non-rigid intraoperative environments, this paper proposes an enhanced multimodal fusion framework—EMF-RSN. This framework achieves spatial consistency between point clouds and intraoperative images through a depth-guided Geometry-Vision Alignment Module (GVAN), implements dynamic weighted fusion of geometric and visual features through a cross-modal attention fusion module (CAFM), and constructs a closed-loop optimization mechanism from perception to decision through a Task Feedback Optimization (TFO) module, thereby improving navigation accuracy and stability.</div><div>Experiments on the public dataset (Hamlyn) and the self-built simulation dataset (Sim-Surgical Fusion) demonstrate that EMF-RSN significantly outperforms existing methods in terms of geometric accuracy, semantic consistency, and task robustness. Compared to traditional registration algorithms, point cloud errors are reduced by approximately 50%, trajectory errors are reduced by over 20%, and real-time performance of 44 FPS is maintained even in complex deformation and occlusion environments. This research provides a new technical approach and model foundation for realizing intelligent surgical navigation that integrates virtual and real elements, and is of great significance for the perception and autonomous control of surgical robots.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"131 ","pages":"Article 104171"},"PeriodicalIF":15.5,"publicationDate":"2026-01-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146014542","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Multimodal spatio-temporal fusion: A generalizable GCN-LSTM with attention framework for urban application 多模态时空融合：一个具有城市应用关注框架的广义GCN-LSTM

IF 15.5 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2026-01-20 DOI: 10.1016/j.inffus.2026.104164

Yunfei Guo

The proliferation of urban big data presents unprecedented opportunities for understanding cities, yet the analytical methods to harness this data are often fragmented and domain-specific. Existing predictive models in urban computing are typically highly specialized, creating analytical silos that inhibit knowledge transfer and are difficult to adapt across domains such as public safety, housing and transport. This paper confronts this critical gap by developing a generalizable, multimodal spatio-temporal deep learning framework engineered for both high predictive performance and interpretability, which is capable of mastering diverse urban prediction tasks without architectural modification. The hybrid architecture fuses a Multi-Head Graph Convolutional Network (GCN) for spatial diffusion, a Long Short-Term Memory (LSTM) network for temporal dynamics, and a learnable Gating Mechanism that weights the influence of spatial graph versus static external features. To validate this generalizability, the framework was tested on three distinct urban domains in London: crime forecasting, housing price estimation and transport network demand. The model outperformed traditional baselines (ARIMA, XGBoost) and state-of-the-art deep learning models (TabNet, TFT). Moreover, the framework moves beyond prediction to explanation by incorporating attention mechanisms and permutation feature importance analysis.

城市大数据的激增为理解城市提供了前所未有的机会，然而利用这些数据的分析方法往往是碎片化的，并且是特定于领域的。城市计算中现有的预测模型通常是高度专业化的，造成了分析孤岛，阻碍了知识的转移，并且难以跨公共安全、住房和交通等领域进行适应。本文通过开发具有高预测性能和可解释性的可推广的多模态时空深度学习框架来解决这一关键差距，该框架能够在不修改架构的情况下掌握各种城市预测任务。该混合架构融合了用于空间扩散的多头图卷积网络（GCN），用于时间动态的长短期记忆（LSTM）网络，以及用于权衡空间图与静态外部特征影响的可学习门控制机制。为了验证这种普遍性，该框架在伦敦三个不同的城市领域进行了测试：犯罪预测、房价估计和交通网络需求。该模型优于传统的基线（ARIMA、XGBoost）和最先进的深度学习模型（TabNet、TFT）。此外，该框架通过结合注意机制和排列特征重要性分析，从预测走向解释。

{"title":"Multimodal spatio-temporal fusion: A generalizable GCN-LSTM with attention framework for urban application","authors":"Yunfei Guo","doi":"10.1016/j.inffus.2026.104164","DOIUrl":"10.1016/j.inffus.2026.104164","url":null,"abstract":"<div><div>The proliferation of urban big data presents unprecedented opportunities for understanding cities, yet the analytical methods to harness this data are often fragmented and domain-specific. Existing predictive models in urban computing are typically highly specialized, creating analytical silos that inhibit knowledge transfer and are difficult to adapt across domains such as public safety, housing and transport. This paper confronts this critical gap by developing a generalizable, multimodal spatio-temporal deep learning framework engineered for both high predictive performance and interpretability, which is capable of mastering diverse urban prediction tasks without architectural modification. The hybrid architecture fuses a Multi-Head Graph Convolutional Network (GCN) for spatial diffusion, a Long Short-Term Memory (LSTM) network for temporal dynamics, and a learnable Gating Mechanism that weights the influence of spatial graph versus static external features. To validate this generalizability, the framework was tested on three distinct urban domains in London: crime forecasting, housing price estimation and transport network demand. The model outperformed traditional baselines (ARIMA, XGBoost) and state-of-the-art deep learning models (TabNet, TFT). Moreover, the framework moves beyond prediction to explanation by incorporating attention mechanisms and permutation feature importance analysis.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"131 ","pages":"Article 104164"},"PeriodicalIF":15.5,"publicationDate":"2026-01-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146014809","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Code-driven programming prediction enhanced by LLM with a feature fusion approach 基于特征融合的LLM增强代码驱动编程预测

IF 15.5 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2026-01-20 DOI: 10.1016/j.inffus.2026.104165

Shengyingjie Liu , Jianxin Li , Qian Wan , Bo He , Zhijun Huang , Qing Li

Programming education is essential for equipping individuals with digital literacy skills and developing the problem-solving abilities necessary for success in the modern workforce. In online programming tutoring systems, knowledge tracing (KT) techniques are crucial for programming prediction, as they monitor user performance and model user cognition. However, both universal and programming-specific knowledge transfer methods depend on traditional state-driven paradigms that indirectly predict programming outcomes based on users’ knowledge states. It does not align with the core objective of programming prediction, which is to determine whether submitted code can solve the question. To address this, we present the code-driven feature fusion KT (CFKT), which integrates large language models (LLM) and encoders for both individualized and common code features. It consists of two modules: pass prediction and code prediction. The pass prediction module leverages LLM to incorporate semantic information from the question and code through embedding, extracting key features that determine code correctness through proxy tasks and effectively narrowing the solution space with vectorization. The code prediction module integrates user historical data and data from other users through feature fusion blocks, allowing for accurate predictions of submitted code and effectively mitigating the cold start problem. Experiments on multiple real-world public programming datasets demonstrate that CFKT significantly outperforms existing baseline methods.

编程教育对于使个人具备数字素养技能和发展在现代劳动力中取得成功所必需的解决问题的能力至关重要。在在线编程辅导系统中，知识跟踪（KT）技术对编程预测至关重要，因为它们监控用户性能并为用户认知建模。然而，通用知识转移方法和特定编程知识转移方法都依赖于传统的状态驱动范式，这些范式基于用户的知识状态间接预测编程结果。它不符合编程预测的核心目标，即确定提交的代码是否可以解决问题。为了解决这个问题，我们提出了代码驱动的特征融合KT (CFKT)，它集成了大型语言模型（LLM）和用于个性化和公共代码特征的编码器。它包括两个模块：传递预测和代码预测。pass预测模块利用LLM通过嵌入将问题和代码中的语义信息结合起来，通过代理任务提取确定代码正确性的关键特征，并通过向量化有效地缩小解决方案空间。代码预测模块通过特征融合块集成用户历史数据和来自其他用户的数据，允许对提交的代码进行准确预测，并有效缓解冷启动问题。在多个真实世界公共编程数据集上的实验表明，CFKT显著优于现有的基线方法。

{"title":"Code-driven programming prediction enhanced by LLM with a feature fusion approach","authors":"Shengyingjie Liu , Jianxin Li , Qian Wan , Bo He , Zhijun Huang , Qing Li","doi":"10.1016/j.inffus.2026.104165","DOIUrl":"10.1016/j.inffus.2026.104165","url":null,"abstract":"<div><div>Programming education is essential for equipping individuals with digital literacy skills and developing the problem-solving abilities necessary for success in the modern workforce. In online programming tutoring systems, knowledge tracing (KT) techniques are crucial for programming prediction, as they monitor user performance and model user cognition. However, both universal and programming-specific knowledge transfer methods depend on traditional state-driven paradigms that indirectly predict programming outcomes based on users’ knowledge states. It does not align with the core objective of programming prediction, which is to determine whether submitted code can solve the question. To address this, we present the code-driven feature fusion KT (CFKT), which integrates large language models (LLM) and encoders for both individualized and common code features. It consists of two modules: pass prediction and code prediction. The pass prediction module leverages LLM to incorporate semantic information from the question and code through embedding, extracting key features that determine code correctness through proxy tasks and effectively narrowing the solution space with vectorization. The code prediction module integrates user historical data and data from other users through feature fusion blocks, allowing for accurate predictions of submitted code and effectively mitigating the cold start problem. Experiments on multiple real-world public programming datasets demonstrate that CFKT significantly outperforms existing baseline methods.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"131 ","pages":"Article 104165"},"PeriodicalIF":15.5,"publicationDate":"2026-01-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146014543","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A novel knowledge distillation method for graph neural networks with gradient mapping and fusion 基于梯度映射和融合的图神经网络知识提取方法

IF 15.5 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2026-01-20 DOI: 10.1016/j.inffus.2026.104163

Kang Liu , Shunzhi Yang , Chang-Dong Wang , Yunwen Chen , Zhenhua Huang

The primary goal of graph knowledge distillation (GKD) is to transfer knowledge from a complex graph neural network (GNN) teacher to a smaller, yet more efficient GNN or multi-layer perceptron student. Although existing methods address network scalability, they rely on a frozen teacher that fails to explain how to derive results, thus limiting performance and hindering the improvement of a student. Therefore, we propose a novel GKD method, termed Dynamic Gradient Distillation (DGD), consisting of Generative Adversarial Imitation Learning (GAIL)-based Gradient Mapping and Two-Stage Gradient Fusion modules. The former builds the teacher’s learning process to understand knowledge by drawing on the principle of GAIL. The latter consists of attention fusion and weighted bias operations. Through the attentional fusion operation, it captures and fuses the responses of the teacher to change the gradient of the student at each layer. The fused gradients are then updated by combining them with the student’s backpropagated gradients using the weighted bias operation. DGD allows the student to inherit and extend the teacher’s learning process efficiently. Extensive experiments conducted with seven publicly available datasets show that DGD could significantly outperform some existing methods in node classification tasks. Our code and data are released at https://github.com/KangL-G/Dynamic-Gradient-Distillation.

图知识蒸馏（GKD）的主要目标是将知识从复杂图神经网络（GNN）教师转移到更小但更高效的GNN或多层感知器学生。虽然现有的方法解决了网络的可扩展性，但它们依赖于一个僵化的老师，无法解释如何得出结果，从而限制了学生的表现，阻碍了学生的进步。因此，我们提出了一种新的GKD方法，称为动态梯度蒸馏（DGD），由基于生成对抗模仿学习（GAIL）的梯度映射和两阶段梯度融合模块组成。前者借鉴了GAIL的原理，构建了教师理解知识的学习过程。后者包括注意融合和加权偏置操作。通过注意融合操作，捕捉并融合教师的反应，改变学生在每一层的梯度。然后使用加权偏置操作将融合的梯度与学生的反向传播梯度结合起来进行更新。DGD允许学生有效地继承和扩展老师的学习过程。在7个公开数据集上进行的大量实验表明，在节点分类任务中，DGD可以显著优于一些现有的方法。我们的代码和数据发布在https://github.com/KangL-G/Dynamic-Gradient-Distillation。

{"title":"A novel knowledge distillation method for graph neural networks with gradient mapping and fusion","authors":"Kang Liu , Shunzhi Yang , Chang-Dong Wang , Yunwen Chen , Zhenhua Huang","doi":"10.1016/j.inffus.2026.104163","DOIUrl":"10.1016/j.inffus.2026.104163","url":null,"abstract":"<div><div>The primary goal of graph knowledge distillation (GKD) is to transfer knowledge from a complex graph neural network (GNN) teacher to a smaller, yet more efficient GNN or multi-layer perceptron student. Although existing methods address network scalability, they rely on a frozen teacher that fails to explain how to derive results, thus limiting performance and hindering the improvement of a student. Therefore, we propose a novel GKD method, termed Dynamic Gradient Distillation (DGD), consisting of Generative Adversarial Imitation Learning (GAIL)-based Gradient Mapping and Two-Stage Gradient Fusion modules. The former builds the teacher’s learning process to understand knowledge by drawing on the principle of GAIL. The latter consists of attention fusion and weighted bias operations. Through the attentional fusion operation, it captures and fuses the responses of the teacher to change the gradient of the student at each layer. The fused gradients are then updated by combining them with the student’s backpropagated gradients using the weighted bias operation. DGD allows the student to inherit and extend the teacher’s learning process efficiently. Extensive experiments conducted with seven publicly available datasets show that DGD could significantly outperform some existing methods in node classification tasks. Our code and data are released at <span><span>https://github.com/KangL-G/Dynamic-Gradient-Distillation</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"131 ","pages":"Article 104163"},"PeriodicalIF":15.5,"publicationDate":"2026-01-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146006526","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Stain-aware domain alignment for imbalance blood cell classification 不平衡血细胞分类的染色敏感区域比对

IF 15.5 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2026-01-20 DOI: 10.1016/j.inffus.2026.104166

Yongcheng Li , Lingcong Cai , Ying Lu , Xiao Han , Ma Li , Wenxing Lai , Xiangzhong Zhang , Xiaomao Fan

Blood cell identification is critical for hematological analysis as it aids physicians in diagnosing various blood-related diseases. In real-world scenarios, blood cell image datasets often present the issues of domain shift and data imbalance, posing challenges for accurate blood cell identification. To address these issues, we propose a novel blood cell classification method termed SADA via stain-aware domain alignment. The primary objective of this work is to mine domain-invariant features in the presence of domain shifts and data imbalance. To accomplish this objective, we propose a stain-based augmentation approach and a local alignment constraint to learn domain-invariant features. Furthermore, we propose a domain-invariant supervised contrastive learning strategy to capture discriminative features. We decouple the training process into two stages of domain-invariant feature learning and classification training, alleviating the problem of data imbalance. Experiment results on four public blood cell datasets and a private real dataset collected from the Third Affiliated Hospital of Sun Yat-sen University demonstrate that SADA can achieve a new state-of-the-art baseline, which is superior to the existing cutting-edge methods. The source code can be available at the URL (https://github.com/AnoK3111/SADA).

血细胞鉴定是血液分析的关键，因为它有助于医生诊断各种血液相关疾病。在现实场景中，血细胞图像数据集经常出现域移位和数据不平衡的问题，这给准确的血细胞识别带来了挑战。为了解决这些问题，我们提出了一种新的血细胞分类方法，称为SADA，通过染色感知结构域对齐。这项工作的主要目标是在存在域移位和数据不平衡的情况下挖掘域不变特征。为了实现这一目标，我们提出了一种基于染色的增强方法和局部对齐约束来学习域不变特征。此外，我们提出了一种领域不变的监督对比学习策略来捕获判别特征。我们将训练过程解耦为域不变特征学习和分类训练两个阶段，缓解了数据不平衡的问题。在中山大学附属第三医院的四个公共血细胞数据集和一个私人真实数据集上的实验结果表明，SADA可以实现新的最先进的基线，优于现有的前沿方法。源代码可以从URL （https://github.com/AnoK3111/SADA）获得。

{"title":"Stain-aware domain alignment for imbalance blood cell classification","authors":"Yongcheng Li , Lingcong Cai , Ying Lu , Xiao Han , Ma Li , Wenxing Lai , Xiangzhong Zhang , Xiaomao Fan","doi":"10.1016/j.inffus.2026.104166","DOIUrl":"10.1016/j.inffus.2026.104166","url":null,"abstract":"<div><div>Blood cell identification is critical for hematological analysis as it aids physicians in diagnosing various blood-related diseases. In real-world scenarios, blood cell image datasets often present the issues of domain shift and data imbalance, posing challenges for accurate blood cell identification. To address these issues, we propose a novel blood cell classification method termed SADA via stain-aware domain alignment. The primary objective of this work is to mine domain-invariant features in the presence of domain shifts and data imbalance. To accomplish this objective, we propose a stain-based augmentation approach and a local alignment constraint to learn domain-invariant features. Furthermore, we propose a domain-invariant supervised contrastive learning strategy to capture discriminative features. We decouple the training process into two stages of domain-invariant feature learning and classification training, alleviating the problem of data imbalance. Experiment results on four public blood cell datasets and a private real dataset collected from the Third Affiliated Hospital of Sun Yat-sen University demonstrate that SADA can achieve a new state-of-the-art baseline, which is superior to the existing cutting-edge methods. The source code can be available at the URL (<span><span>https://github.com/AnoK3111/SADA</span><svg><path></path></svg></span>).</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"131 ","pages":"Article 104166"},"PeriodicalIF":15.5,"publicationDate":"2026-01-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146014547","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Validity-aware context modeling for gradient-guided image inpainting 基于有效性感知的梯度引导图像绘制上下文建模

IF 15.5 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2026-01-19 DOI: 10.1016/j.inffus.2026.104162

Wuzhen Shi , Wu Yang , Zhihao Wu , Yang Wen

Existing prior-guided image inpainting methods show state-of-the-art performance. But their prior extraction is computationally expensive and unstable in accuracy. Besides, most of them only focus on the structure guidance, which hardly facilitates the repair of realistic textures. Inspired by the fact that gradient maps are easy to extract and reflect both image structure and fine texture details, this paper proposes a gradient-guided network for image inpainting, which first uses the gradient context information and multi-level image compensation features to repair the gradient, and then uses the repaired gradient features to guide the generation of realistic image. A gradient-driven attention (GDA) module is introduced for efficient prior guidance. Additionally, a context validity-aware (CVA) module is proposed for progressively filling hole regions of images, which accurately utilizes both local and contextual information for image inpainting via validity-aware measurements. Furthermore, by artificially manipulating the generation of the gradient map, our gradient-guided image inpainting method enables user-guided image editing, which effectively increases the diversity of image generation and enhances the flexibility of image editing. Experiments on benchmark datasets show that the proposed method outperforms the state-of-the-art methods. Extensive ablation experiments are also conducted to demonstrate the effectiveness of each module.

现有的先验引导图像绘制方法显示了最先进的性能。但它们的先验提取计算成本高，精度不稳定。此外，它们大多只注重结构引导，难以实现逼真纹理的修复。基于梯度图易于提取和反映图像结构和精细纹理细节的特点，本文提出了一种用于图像补图的梯度引导网络，该网络首先利用梯度上下文信息和多级图像补偿特征对梯度进行修复，然后利用修复后的梯度特征指导生成逼真的图像。引入梯度驱动注意力（GDA）模块，实现有效的事前引导。此外，提出了一种上下文有效性感知（CVA）模块，用于逐步填充图像的空洞区域，该模块通过有效性感知测量准确地利用局部和上下文信息进行图像绘制。此外，我们的梯度引导图像绘制方法通过人为操纵梯度图的生成，实现用户引导的图像编辑，有效地增加了图像生成的多样性，增强了图像编辑的灵活性。在基准数据集上的实验表明，该方法优于现有的方法。为了验证每个模块的有效性，还进行了大量的烧蚀实验。

{"title":"Validity-aware context modeling for gradient-guided image inpainting","authors":"Wuzhen Shi , Wu Yang , Zhihao Wu , Yang Wen","doi":"10.1016/j.inffus.2026.104162","DOIUrl":"10.1016/j.inffus.2026.104162","url":null,"abstract":"<div><div>Existing prior-guided image inpainting methods show state-of-the-art performance. But their prior extraction is computationally expensive and unstable in accuracy. Besides, most of them only focus on the structure guidance, which hardly facilitates the repair of realistic textures. Inspired by the fact that gradient maps are easy to extract and reflect both image structure and fine texture details, this paper proposes a gradient-guided network for image inpainting, which first uses the gradient context information and multi-level image compensation features to repair the gradient, and then uses the repaired gradient features to guide the generation of realistic image. A gradient-driven attention (GDA) module is introduced for efficient prior guidance. Additionally, a context validity-aware (CVA) module is proposed for progressively filling hole regions of images, which accurately utilizes both local and contextual information for image inpainting via validity-aware measurements. Furthermore, by artificially manipulating the generation of the gradient map, our gradient-guided image inpainting method enables user-guided image editing, which effectively increases the diversity of image generation and enhances the flexibility of image editing. Experiments on benchmark datasets show that the proposed method outperforms the state-of-the-art methods. Extensive ablation experiments are also conducted to demonstrate the effectiveness of each module.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"131 ","pages":"Article 104162"},"PeriodicalIF":15.5,"publicationDate":"2026-01-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146000897","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Data fusion for low-cost sensors: A systematic literature review 低成本传感器的数据融合：系统文献综述

IF 15.5 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2026-01-18 DOI: 10.1016/j.inffus.2026.104124

Gabriel Oduori , Chaira Cocco , Payam Sajadi , Francesco Pilla

Data fusion (DF) addresses the challenge of integrating heterogeneous data sources to improve decision-making and inference. Although DF has been widely explored, no prior systematic review has specifically focused on its application to low-cost sensor (LCS) data in environmental monitoring. To address this gap, we conduct a systematic literature review (SLR) following the PRISMA framework, synthesising findings from 82 peer-reviewed articles. The review addresses three key questions: (1) What fusion methodologies are employed in conjunction with LCS data? (2) In what environmental contexts are these methods applied? (3) What are the methodological challenges and research gaps? Our analysis reveals that geostatistical and machine learning approaches dominate current practice, with air quality monitoring emerging as the primary application domain. Additionally, artificial intelligence (AI)-based methods are increasingly used to integrate spatial, temporal, and multimodal data. However, limitations persist in uncertainty quantification, validation standards, and the generalisability of fusion frameworks. This review provides a comprehensive synthesis of current techniques and outlines key directions for future research, including the development of robust, uncertainty-aware fusion methods and broader application to less-studied environmental variables.

数据融合（DF）解决了集成异构数据源以改进决策和推理的挑战。虽然DF已被广泛探索，但尚未有系统综述专门关注其在环境监测中低成本传感器（LCS）数据中的应用。为了解决这一差距，我们根据PRISMA框架进行了系统的文献综述（SLR），综合了82篇同行评议文章的发现。该综述解决了三个关键问题：(1)结合LCS数据采用了哪些融合方法？(2)这些方法适用于什么环境背景？(3)方法论上的挑战和研究差距是什么？我们的分析表明，地质统计学和机器学习方法主导了当前的实践，空气质量监测正在成为主要的应用领域。此外，基于人工智能（AI）的方法越来越多地用于整合空间、时间和多模态数据。然而，在不确定度量化、验证标准和融合框架的通用性方面仍然存在局限性。这篇综述提供了对当前技术的全面综合，并概述了未来研究的关键方向，包括鲁棒性、不确定性感知融合方法的发展以及对较少研究的环境变量的更广泛应用。

{"title":"Data fusion for low-cost sensors: A systematic literature review","authors":"Gabriel Oduori , Chaira Cocco , Payam Sajadi , Francesco Pilla","doi":"10.1016/j.inffus.2026.104124","DOIUrl":"10.1016/j.inffus.2026.104124","url":null,"abstract":"<div><div>Data fusion (DF) addresses the challenge of integrating heterogeneous data sources to improve decision-making and inference. Although DF has been widely explored, no prior systematic review has specifically focused on its application to low-cost sensor (LCS) data in environmental monitoring. To address this gap, we conduct a systematic literature review (SLR) following the PRISMA framework, synthesising findings from 82 peer-reviewed articles. The review addresses three key questions: (1) What fusion methodologies are employed in conjunction with LCS data? (2) In what environmental contexts are these methods applied? (3) What are the methodological challenges and research gaps? Our analysis reveals that geostatistical and machine learning approaches dominate current practice, with air quality monitoring emerging as the primary application domain. Additionally, artificial intelligence (AI)-based methods are increasingly used to integrate spatial, temporal, and multimodal data. However, limitations persist in uncertainty quantification, validation standards, and the generalisability of fusion frameworks. This review provides a comprehensive synthesis of current techniques and outlines key directions for future research, including the development of robust, uncertainty-aware fusion methods and broader application to less-studied environmental variables.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"131 ","pages":"Article 104124"},"PeriodicalIF":15.5,"publicationDate":"2026-01-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145995206","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Speech emotion recognition: A systematic mega-review of techniques and pipelines 语音情感识别：技术和管道的系统综述

IF 15.5 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Information Fusion

Pub Date : 2026-01-18 DOI: 10.1016/j.inffus.2026.104161

Adil Chakhtouna, Sara Sekkate, Abdellah Adib

Speech Emotion Recognition (SER) is a rapidly evolving research field that aims to enable machines to automatically identify human emotions from vocal signals. This systematic review presents a comprehensive and structured synthesis of the SER literature, focusing on eleven key research questions that span the theoretical foundations, signal processing pipeline, and methodological advancements in the field. Unlike prior surveys, this review unifies both foundational and state-of-the-art insights across all stages of the SER pipeline through a question-driven structure, offering a clear road-map for both new and experienced researchers in the SER community. We first explore the psychological and computational modeling of emotions, followed by a detailed examination of the different modalities for emotion expression, with a particular emphasis on speech. The review highlights the most widely used emotional speech databases, common pre-processing techniques, and the diverse set of handcrafted and learned features employed in SER. We compare traditional machine learning approaches with recent deep learning models, emphasizing their respective strengths, limitations, and application contexts. Special attention is given to the recent shift toward self-supervised learning (SSL) models such as Wav2Vec2 and HuBERT, which have redefined the state-of-the-art in speech-based representation learning. Special attention is given to evaluation metrics, benchmarking strategies, and real-world deployment challenges, including issues of speaker-independence and environmental variability. The review concludes by identifying key limitations across the literature and articulating future research directions necessary for developing reliable, scalable, and context-aware emotion-aware systems. Overall, this work serves as a central reference for advancing SER research and practical deployment in real-world environments.

语音情感识别（SER）是一个快速发展的研究领域，旨在使机器能够从声音信号中自动识别人类的情感。本系统综述对SER文献进行了全面和结构化的综合，重点关注11个关键研究问题，这些问题涵盖了该领域的理论基础、信号处理管道和方法进展。与之前的调查不同，该综述通过一个问题驱动的结构，统一了SER管道所有阶段的基础和最新的见解，为SER社区的新研究人员和经验丰富的研究人员提供了清晰的路线图。我们首先探索情绪的心理和计算建模，然后详细检查情绪表达的不同方式，特别强调语言。这篇综述强调了最广泛使用的情绪语音数据库，常见的预处理技术，以及SER中使用的各种手工和学习特征。我们将传统的机器学习方法与最近的深度学习模型进行比较，强调它们各自的优势、局限性和应用环境。特别关注最近向自我监督学习（SSL）模型的转变，如Wav2Vec2和HuBERT，它们重新定义了基于语音的表示学习的最新技术。特别关注评估指标、基准策略和现实世界的部署挑战，包括说话者独立性和环境可变性问题。本文总结了文献中的关键限制，并阐明了开发可靠、可扩展和上下文感知的情感感知系统所需的未来研究方向。总的来说，这项工作是推进SER研究和在现实环境中实际部署的核心参考。

{"title":"Speech emotion recognition: A systematic mega-review of techniques and pipelines","authors":"Adil Chakhtouna, Sara Sekkate, Abdellah Adib","doi":"10.1016/j.inffus.2026.104161","DOIUrl":"10.1016/j.inffus.2026.104161","url":null,"abstract":"<div><div>Speech Emotion Recognition (SER) is a rapidly evolving research field that aims to enable machines to automatically identify human emotions from vocal signals. This systematic review presents a comprehensive and structured synthesis of the SER literature, focusing on eleven key research questions that span the theoretical foundations, signal processing pipeline, and methodological advancements in the field. Unlike prior surveys, this review unifies both foundational and state-of-the-art insights across all stages of the SER pipeline through a question-driven structure, offering a clear road-map for both new and experienced researchers in the SER community. We first explore the psychological and computational modeling of emotions, followed by a detailed examination of the different modalities for emotion expression, with a particular emphasis on speech. The review highlights the most widely used emotional speech databases, common pre-processing techniques, and the diverse set of handcrafted and learned features employed in SER. We compare traditional machine learning approaches with recent deep learning models, emphasizing their respective strengths, limitations, and application contexts. Special attention is given to the recent shift toward self-supervised learning (SSL) models such as Wav2Vec2 and HuBERT, which have redefined the state-of-the-art in speech-based representation learning. Special attention is given to evaluation metrics, benchmarking strategies, and real-world deployment challenges, including issues of speaker-independence and environmental variability. The review concludes by identifying key limitations across the literature and articulating future research directions necessary for developing reliable, scalable, and context-aware emotion-aware systems. Overall, this work serves as a central reference for advancing SER research and practical deployment in real-world environments.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"131 ","pages":"Article 104161"},"PeriodicalIF":15.5,"publicationDate":"2026-01-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145995202","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0