Pub Date : 2025-01-18DOI: 10.1016/j.inffus.2024.102914
Xiwei Shen, Jie Kong, Yang Song, Xinyi Wang, Grant Mosey
In Las Vegas and many other desert cities, the unique climatic conditions, marked by high daytime temperatures, naturally encourage residents to seek outdoor recreational activities during the cooler evening hours. However, the approach to streetlight management has been less than optimal, leading to inadequate illumination in public parks after dark. This lack of proper lighting compromises not only the safety but also the enjoyment opportunity of these spaces during the night, a time when they could offer a much-needed respite during summer heat. Recent scholarship has highlighted the deterrence of park usage due to poor design of the street lighting, pointing to a broader issue in urban planning that requires attention to adapt infrastructures to local climates for the benefit of public health and well-being. This study seeks to contribute to the existing scholarship on park lighting by utilizing diverse data sources and creating longitudinal measures to examine how population behaviors in urban parks vary over time in different locations. It seeks to explore the impact of park users’ demographics, particularly variations across race and income levels, and the density of street lighting on the nighttime usage of public green spaces by using the time fixed effect method. It aims to understand how demographic diversity among park users and the physical environment, specifically street lighting density, influences patterns of nighttime activities in public parks. Using this analysis, we develop an improved predictive model for determining the density of street lighting in public green spaces by comparing multiple types of machine learning models. This model will consider the demographic diversity of users and the observed patterns of nighttime usage, with the goal of enhancing accessibility, safety, and utilization of these spaces during nighttime hours. The significance of this research contributes to the broader objective of creating resilient, healthy, and inclusive cities that cater to the well-being of their residents.
{"title":"Optimizing the environmental design and management of public green spaces: Analyzing urban infrastructure and long-term user experience with a focus on streetlight density in the city of Las Vegas, NV","authors":"Xiwei Shen, Jie Kong, Yang Song, Xinyi Wang, Grant Mosey","doi":"10.1016/j.inffus.2024.102914","DOIUrl":"https://doi.org/10.1016/j.inffus.2024.102914","url":null,"abstract":"In Las Vegas and many other desert cities, the unique climatic conditions, marked by high daytime temperatures, naturally encourage residents to seek outdoor recreational activities during the cooler evening hours. However, the approach to streetlight management has been less than optimal, leading to inadequate illumination in public parks after dark. This lack of proper lighting compromises not only the safety but also the enjoyment opportunity of these spaces during the night, a time when they could offer a much-needed respite during summer heat. Recent scholarship has highlighted the deterrence of park usage due to poor design of the street lighting, pointing to a broader issue in urban planning that requires attention to adapt infrastructures to local climates for the benefit of public health and well-being. This study seeks to contribute to the existing scholarship on park lighting by utilizing diverse data sources and creating longitudinal measures to examine how population behaviors in urban parks vary over time in different locations. It seeks to explore the impact of park users’ demographics, particularly variations across race and income levels, and the density of street lighting on the nighttime usage of public green spaces by using the time fixed effect method. It aims to understand how demographic diversity among park users and the physical environment, specifically street lighting density, influences patterns of nighttime activities in public parks. Using this analysis, we develop an improved predictive model for determining the density of street lighting in public green spaces by comparing multiple types of machine learning models. This model will consider the demographic diversity of users and the observed patterns of nighttime usage, with the goal of enhancing accessibility, safety, and utilization of these spaces during nighttime hours. The significance of this research contributes to the broader objective of creating resilient, healthy, and inclusive cities that cater to the well-being of their residents.","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"102 1","pages":""},"PeriodicalIF":18.6,"publicationDate":"2025-01-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142990595","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Accurate and robust road extraction with good continuity and completeness is crucial for the development of smart city and intelligent transportation. Remote sensing images and vehicle trajectories are attractive data sources with rich and complementary multimodal road information, and the fusion of them promises to significantly promote the performance of road extraction. However, existing studies on fusion-based road extraction suffer from the problems that the feature extraction modules pay little attention to the inherent morphology of roads, and the multimodal feature fusion techniques are too simple and superficial to fully and efficiently exploit the complementary information from different data sources, resulting in road predictions with poor continuity and limited performance. To this end, we propose a Bilateral Synergistic Fusion network with novel Dynamic Flow convolution, termed DF-BSFNet, which fully leverages the complementary road information from images and trajectories in a dual-mutual adaptive guidance and incremental refinement manner. First, we propose a novel Dynamic Flow Convolution (DFConv) that more adeptly and consciously captures the elongated and winding “flow” morphology of roads in complex scenarios, providing flexible and powerful capabilities for learning detail-heavy and robust road feature representations. Second, we develop two parallel modality-specific feature extractors with DFConv to extract hierarchical road features specific to images and trajectories, effectively exploiting the distinctive advantages of each modality. Third, we propose a Bilateral Synergistic Adaptive Feature Fusion (BSAFF) module which synthesizes the global-context and local-context of complementary multimodal road information and achieves a sophisticated feature fusion with dynamic guided-propagation and dual-mutual refinement. Extensive experiments on three road datasets demonstrate that our DF-BSFNet outperforms current state-of-the-art methods by a large margin in terms of continuity and accuracy.
{"title":"DF-BSFNet: A bilateral synergistic fusion network with novel dynamic flow convolution for robust road extraction","authors":"Chong Zhang, Huazu Zhang, Xiaogang Guo, Heng Qi, Zilong Zhao, Luliang Tang","doi":"10.1016/j.inffus.2025.102958","DOIUrl":"https://doi.org/10.1016/j.inffus.2025.102958","url":null,"abstract":"Accurate and robust road extraction with good continuity and completeness is crucial for the development of smart city and intelligent transportation. Remote sensing images and vehicle trajectories are attractive data sources with rich and complementary multimodal road information, and the fusion of them promises to significantly promote the performance of road extraction. However, existing studies on fusion-based road extraction suffer from the problems that the feature extraction modules pay little attention to the inherent morphology of roads, and the multimodal feature fusion techniques are too simple and superficial to fully and efficiently exploit the complementary information from different data sources, resulting in road predictions with poor continuity and limited performance. To this end, we propose a <ce:bold>B</ce:bold>ilateral <ce:bold>S</ce:bold>ynergistic <ce:bold>F</ce:bold>usion network with novel <ce:bold>D</ce:bold>ynamic <ce:bold>F</ce:bold>low convolution, termed DF-BSFNet, which fully leverages the complementary road information from images and trajectories in a dual-mutual adaptive guidance and incremental refinement manner. First, we propose a novel Dynamic Flow Convolution (DFConv) that more adeptly and consciously captures the elongated and winding “flow” morphology of roads in complex scenarios, providing flexible and powerful capabilities for learning detail-heavy and robust road feature representations. Second, we develop two parallel modality-specific feature extractors with DFConv to extract hierarchical road features specific to images and trajectories, effectively exploiting the distinctive advantages of each modality. Third, we propose a Bilateral Synergistic Adaptive Feature Fusion (BSAFF) module which synthesizes the global-context and local-context of complementary multimodal road information and achieves a sophisticated feature fusion with dynamic guided-propagation and dual-mutual refinement. Extensive experiments on three road datasets demonstrate that our DF-BSFNet outperforms current state-of-the-art methods by a large margin in terms of continuity and accuracy.","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"9 1","pages":""},"PeriodicalIF":18.6,"publicationDate":"2025-01-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142990593","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
To enhance the comprehensiveness of fusion features and meet the requirements of high-level vision tasks, some fusion methods attempt to coordinate the fusion process by directly interacting with the high-level semantic feature. However, due to the significant disparity between high-level semantic domain and fusion representation domain, there is potential for enhancing the effectiveness of the collaborative approach to direct interaction. To overcome this obstacle, a high-level vision task-driven infrared and visible image fusion method based on cross-domain knowledge distillation is proposed, referred to as KDFuse. The KDFuse brings multi-task perceptual representation into the same domain through cross-domain knowledge distillation. By facilitating interaction between semantic information and fusion information at an equivalent level, it effectively reduces the gap between the semantic and fusion domains, enabling multi-task collaborative fusion. Specifically, to acquire superior high-level semantic representations essential for instructing the fusion network, the teaching relationship is established to realize multi-task collaboration by the multi-domain interaction distillation module (MIDM). The multi-scale semantic perception module (MSPM) is designed to learn the ability to capture semantic information through the cross-domain knowledge distillation and the semantic detail integration module (SDIM) is constructed to integrate the fusion-level semantic representations with the fusion-level visual representations. Moreover, to balance the semantic and visual representations during the fusion process, the Fourier transform is introduced into the loss function. Extensive comprehensive experiments demonstrate the effectiveness of the proposed method in both image fusion and downstream tasks. The source code is available at https://github.com/lxq-jnu/KDFuse.
{"title":"KDFuse: A high-level vision task-driven infrared and visible image fusion method based on cross-domain knowledge distillation","authors":"Chenjia Yang, Xiaoqing Luo, Zhancheng Zhang, Zhiguo Chen, Xiao-jun Wu","doi":"10.1016/j.inffus.2025.102944","DOIUrl":"https://doi.org/10.1016/j.inffus.2025.102944","url":null,"abstract":"To enhance the comprehensiveness of fusion features and meet the requirements of high-level vision tasks, some fusion methods attempt to coordinate the fusion process by directly interacting with the high-level semantic feature. However, due to the significant disparity between high-level semantic domain and fusion representation domain, there is potential for enhancing the effectiveness of the collaborative approach to direct interaction. To overcome this obstacle, a high-level vision task-driven infrared and visible image fusion method based on cross-domain knowledge distillation is proposed, referred to as KDFuse. The KDFuse brings multi-task perceptual representation into the same domain through cross-domain knowledge distillation. By facilitating interaction between semantic information and fusion information at an equivalent level, it effectively reduces the gap between the semantic and fusion domains, enabling multi-task collaborative fusion. Specifically, to acquire superior high-level semantic representations essential for instructing the fusion network, the teaching relationship is established to realize multi-task collaboration by the multi-domain interaction distillation module (MIDM). The multi-scale semantic perception module (MSPM) is designed to learn the ability to capture semantic information through the cross-domain knowledge distillation and the semantic detail integration module (SDIM) is constructed to integrate the fusion-level semantic representations with the fusion-level visual representations. Moreover, to balance the semantic and visual representations during the fusion process, the Fourier transform is introduced into the loss function. Extensive comprehensive experiments demonstrate the effectiveness of the proposed method in both image fusion and downstream tasks. The source code is available at <ce:inter-ref xlink:href=\"https://github.com/lxq-jnu/KDFuse\" xlink:type=\"simple\">https://github.com/lxq-jnu/KDFuse</ce:inter-ref>.","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"31 1","pages":""},"PeriodicalIF":18.6,"publicationDate":"2025-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142990592","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-10DOI: 10.1016/j.inffus.2025.102932
Neha Singh, Mainak Adhikari
Federated learning (FL) offers a decentralized and collaborative training solution on resource-constraint Edge Devices (EDs) to improve a global model without sharing raw data. Standard Synchronous FL (SFL) approaches provide significant advantages in terms of data privacy and reduced communication overhead, however, face several challenges including Non-independent and identically distributed (Non-IID) data, the presence of unlabeled data, biased aggregation due to device heterogeneity and effective EDs selection to handle the straggler. To tackle these challenges, we propose a new Self-adaptive Federated Learning (SelfFed) strategy using a masked loss function to handle unlabeled data. This allows EDs to concentrate on labeled data, enhancing training efficiency. Additionally, we integrate a novel quality-dependent aggregation solution to mitigate bias during model updates through aggregation. This solution accurately reflects performance across Non-IID data distributions by incentivizing local EDs using a new Stackelberg game model. The model provides rewards based on their contributions to the global model, thereby keeping the EDs motivated to participate and perform well. Finally, we incorporate a deep reinforcement learning technique into the proposed SelfFed strategy for dynamic ED selection to handle straggler EDs. This technique adapts to changes in device performance and resources over iterations, fostering collaboration and sustained engagement. The performance of the SelfFed strategy is evaluated using a real-time SFL scenario (irrigation control in paddy fields) and three benchmark datasets using a serverless private cloud environment. Comparative results against state-of-the-art approaches reveal that the SelfFed significantly reduces CPU usage by 5%–6% and enhances training efficiency by 4%–8% while achieving 4%–6% higher accuracy. Further, in the real-time scenario, the SelfFed improves CPU usage by 3%–5% and enhances training efficiency by 8%–10% with 5%–7% higher accuracy.
{"title":"SelfFed: Self-adaptive Federated Learning with Non-IID data on Heterogeneous Edge Devices for Bias Mitigation and Enhance Training Efficiency","authors":"Neha Singh, Mainak Adhikari","doi":"10.1016/j.inffus.2025.102932","DOIUrl":"https://doi.org/10.1016/j.inffus.2025.102932","url":null,"abstract":"Federated learning (FL) offers a decentralized and collaborative training solution on resource-constraint Edge Devices (EDs) to improve a global model without sharing raw data. Standard Synchronous FL (SFL) approaches provide significant advantages in terms of data privacy and reduced communication overhead, however, face several challenges including Non-independent and identically distributed (Non-IID) data, the presence of unlabeled data, biased aggregation due to device heterogeneity and effective EDs selection to handle the straggler. To tackle these challenges, we propose a new Self-adaptive Federated Learning (SelfFed) strategy using a masked loss function to handle unlabeled data. This allows EDs to concentrate on labeled data, enhancing training efficiency. Additionally, we integrate a novel quality-dependent aggregation solution to mitigate bias during model updates through aggregation. This solution accurately reflects performance across Non-IID data distributions by incentivizing local EDs using a new Stackelberg game model. The model provides rewards based on their contributions to the global model, thereby keeping the EDs motivated to participate and perform well. Finally, we incorporate a deep reinforcement learning technique into the proposed SelfFed strategy for dynamic ED selection to handle straggler EDs. This technique adapts to changes in device performance and resources over iterations, fostering collaboration and sustained engagement. The performance of the SelfFed strategy is evaluated using a real-time SFL scenario (irrigation control in paddy fields) and three benchmark datasets using a serverless private cloud environment. Comparative results against state-of-the-art approaches reveal that the SelfFed significantly reduces CPU usage by 5%–6% and enhances training efficiency by 4%–8% while achieving 4%–6% higher accuracy. Further, in the real-time scenario, the SelfFed improves CPU usage by 3%–5% and enhances training efficiency by 8%–10% with 5%–7% higher accuracy.","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"28 1","pages":""},"PeriodicalIF":18.6,"publicationDate":"2025-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142975154","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-09DOI: 10.1016/j.inffus.2024.102924
Chengyue Wang, Haicheng Liao, Kaiqun Zhu, Guohui Zhang, Zhenning Li
Autonomous vehicles (AVs) rely on accurate trajectory prediction of surrounding vehicles to ensure the safety of both passengers and other road users. Trajectory prediction spans both short-term and long-term horizons, each requiring distinct considerations: short-term predictions rely on accurately capturing the vehicle’s dynamics, while long-term predictions rely on accurately modeling the interaction patterns within the environment. However current approaches, either physics-based or learning-based models, always ignore these distinct considerations, making them struggle to find the optimal prediction for both short-term and long-term horizon. In this paper, we introduce the Dynamics-Enhanced Learning MOdel (DEMO), a novel approach that combines a physics-based Vehicle Dynamics Model with advanced deep learning algorithms. DEMO employs a two-stage architecture, featuring a Dynamics Learning Stage and an Interaction Learning Stage, where the former stage focuses on capturing vehicle motion dynamics and the latter focuses on modeling interaction. By capitalizing on the respective strengths of both methods, DEMO facilitates multi-horizon predictions for future trajectories. Experimental results on the Next Generation Simulation (NGSIM), Macau Connected Autonomous Driving (MoCAD), Highway Drone (HighD), and nuScenes datasets demonstrate that DEMO outperforms state-of-the-art (SOTA) baselines in both short-term and long-term prediction horizons.
{"title":"DEMO: A Dynamics-Enhanced Learning Model for multi-horizon trajectory prediction in autonomous vehicles","authors":"Chengyue Wang, Haicheng Liao, Kaiqun Zhu, Guohui Zhang, Zhenning Li","doi":"10.1016/j.inffus.2024.102924","DOIUrl":"https://doi.org/10.1016/j.inffus.2024.102924","url":null,"abstract":"Autonomous vehicles (AVs) rely on accurate trajectory prediction of surrounding vehicles to ensure the safety of both passengers and other road users. Trajectory prediction spans both short-term and long-term horizons, each requiring distinct considerations: short-term predictions rely on accurately capturing the vehicle’s dynamics, while long-term predictions rely on accurately modeling the interaction patterns within the environment. However current approaches, either physics-based or learning-based models, always ignore these distinct considerations, making them struggle to find the optimal prediction for both short-term and long-term horizon. In this paper, we introduce the <ce:bold>D</ce:bold>ynamics-<ce:bold>E</ce:bold>nhanced Learning <ce:bold>MO</ce:bold>del (<ce:bold>DEMO</ce:bold>), a novel approach that combines a physics-based Vehicle Dynamics Model with advanced deep learning algorithms. DEMO employs a two-stage architecture, featuring a Dynamics Learning Stage and an Interaction Learning Stage, where the former stage focuses on capturing vehicle motion dynamics and the latter focuses on modeling interaction. By capitalizing on the respective strengths of both methods, DEMO facilitates multi-horizon predictions for future trajectories. Experimental results on the Next Generation Simulation (NGSIM), Macau Connected Autonomous Driving (MoCAD), Highway Drone (HighD), and nuScenes datasets demonstrate that DEMO outperforms state-of-the-art (SOTA) baselines in both short-term and long-term prediction horizons.","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"29 1","pages":""},"PeriodicalIF":18.6,"publicationDate":"2025-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142974947","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Utilizing multi-view learning to analyze fish behavior is crucial for fish disease early warning and developing intelligent feeding strategies. Trusted multi-view classification based on Dempster–Shafer Theory (DST) can effectively resolve view conflicts and significantly improve accuracy. However, these DST-based methods often assume that view source domain data are “independent”, and ignore the associations between different views, this can lead to inaccurate fusion and decision errors. To address this limitation, this paper proposes a Trusted Multi-View Fish (TMVF) Behavior Recognition Model that leverages adaptive fusion of associative feature evidence. TMVF employs a Multi-Source Composite Backbone (MSCB) at the feature level to integrate learning across different visual feature dimensions, providing non-independent feature vectors for deeper associative distribution learning. Additionally, a Trusted Association Multi-view (TAMV) Feature Fusion Module is introduced at the vector evidence level. TAMV utilizes a cross-association fusion method to capture the deeper associations between feature vectors rather than treating them as independent sources. It also employs a Dirichlet distribution for more reliable predictions, addressing conflicts between views. To validate TMVF’s performance, a real-world Multi-view Fish Behavior Recognition Dataset (MFBR) with top, underwater, and depth color views was constructed. Experimental results demonstrated TAMV’s superior performance on both the SynDD2 and MFBR datasets. Notably, TMVF achieved an accuracy of 98.48% on SynDD2, surpassing the Frame-flexible network (FFN) by 9.94%. On the MFBR dataset, TMVF achieved an accuracy of 96.56% and an F1-macro score of 94.31%, outperforming I3d+resnet50 by 10.62% and 50.4%, and the FFN by 4.5% and 30.58%, respectively. This demonstrates the effectiveness of TMVF in multi view tasks such as human and animal behavior recognition. The code will be publicly available on GitHub (https://github.com/crazysboy/TMVF).
{"title":"TMVF: Trusted Multi-View Fish Behavior Recognition with correlative feature and adaptive evidence fusion","authors":"Zhenxi Zhao, Xinting Yang, Chunjiang Zhao, Chao Zhou","doi":"10.1016/j.inffus.2024.102899","DOIUrl":"https://doi.org/10.1016/j.inffus.2024.102899","url":null,"abstract":"Utilizing multi-view learning to analyze fish behavior is crucial for fish disease early warning and developing intelligent feeding strategies. Trusted multi-view classification based on Dempster–Shafer Theory (DST) can effectively resolve view conflicts and significantly improve accuracy. However, these DST-based methods often assume that view source domain data are “independent”, and ignore the associations between different views, this can lead to inaccurate fusion and decision errors. To address this limitation, this paper proposes a Trusted Multi-View Fish (TMVF) Behavior Recognition Model that leverages adaptive fusion of associative feature evidence. TMVF employs a Multi-Source Composite Backbone (MSCB) at the feature level to integrate learning across different visual feature dimensions, providing non-independent feature vectors for deeper associative distribution learning. Additionally, a Trusted Association Multi-view (TAMV) Feature Fusion Module is introduced at the vector evidence level. TAMV utilizes a cross-association fusion method to capture the deeper associations between feature vectors rather than treating them as independent sources. It also employs a Dirichlet distribution for more reliable predictions, addressing conflicts between views. To validate TMVF’s performance, a real-world Multi-view Fish Behavior Recognition Dataset (MFBR) with top, underwater, and depth color views was constructed. Experimental results demonstrated TAMV’s superior performance on both the SynDD2 and MFBR datasets. Notably, TMVF achieved an accuracy of 98.48% on SynDD2, surpassing the Frame-flexible network (FFN) by 9.94%. On the MFBR dataset, TMVF achieved an accuracy of 96.56% and an F1-macro score of 94.31%, outperforming I3d+resnet50 by 10.62% and 50.4%, and the FFN by 4.5% and 30.58%, respectively. This demonstrates the effectiveness of TMVF in multi view tasks such as human and animal behavior recognition. The code will be publicly available on GitHub (<ce:inter-ref xlink:href=\"https://github.com/crazysboy/TMVF\" xlink:type=\"simple\">https://github.com/crazysboy/TMVF</ce:inter-ref>).","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"205 1","pages":""},"PeriodicalIF":18.6,"publicationDate":"2025-01-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142974949","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-08DOI: 10.1016/j.inffus.2025.102934
Qi Zhang, Mingfei Lu, Jingmin Xin, Badong Chen
Efficiently preserving task-relevant information while removing noise and redundancy in multi-view data remains a core challenge. The information bottleneck principle offers an information-theoretic framework to compress data while retaining essential information for the task. However, estimating mutual information in high-dimensional spaces is computationally intractable. Commonly used variational methods introduce uncertainty and risk performance degradation. To overcome these limitations, we propose a robust deterministic multi-view information bottleneck framework that circumvents the need for variational inference or distributional assumptions. Specifically, we present a non-parametric mutual information estimation based on the Cauchy–Schwarz divergence, eliminating the need for auxiliary neural estimators and significantly simplifying the optimization of the information bottleneck. Leveraging this mutual information measure, we design a neural network framework that robustly compresses high-dimensional multi-view data into a low-dimensional representation, extracting task-relevant features that adhere to both sufficiency and minimality. Additionally, attention mechanisms are employed to fuse compact features across different views, capturing interdependencies and enhancing the integration of complementary information. This fusion process improves the robustness of the overall representation. Statistical analysis using the Nemenyi test shows statistically significant differences in performance between our method and existing algorithms, with a critical distance (CD = 1.856, p-value <0.05), demonstrating the superiority of our approach. Experimental results on synthetic data highlight the framework’s robustness in handling noise and redundancy, demonstrating its effectiveness in challenging environments. Validation on eight real-world datasets, including electroencephalography and Alzheimer’s neuroimaging data, confirms its superior performance, particularly with limited training samples. The implementation is available at https://github.com/archy666/CSMVIB.
{"title":"Towards a robust multi-view information bottleneck using Cauchy–Schwarz divergence","authors":"Qi Zhang, Mingfei Lu, Jingmin Xin, Badong Chen","doi":"10.1016/j.inffus.2025.102934","DOIUrl":"https://doi.org/10.1016/j.inffus.2025.102934","url":null,"abstract":"Efficiently preserving task-relevant information while removing noise and redundancy in multi-view data remains a core challenge. The information bottleneck principle offers an information-theoretic framework to compress data while retaining essential information for the task. However, estimating mutual information in high-dimensional spaces is computationally intractable. Commonly used variational methods introduce uncertainty and risk performance degradation. To overcome these limitations, we propose a robust deterministic multi-view information bottleneck framework that circumvents the need for variational inference or distributional assumptions. Specifically, we present a non-parametric mutual information estimation based on the Cauchy–Schwarz divergence, eliminating the need for auxiliary neural estimators and significantly simplifying the optimization of the information bottleneck. Leveraging this mutual information measure, we design a neural network framework that robustly compresses high-dimensional multi-view data into a low-dimensional representation, extracting task-relevant features that adhere to both sufficiency and minimality. Additionally, attention mechanisms are employed to fuse compact features across different views, capturing interdependencies and enhancing the integration of complementary information. This fusion process improves the robustness of the overall representation. Statistical analysis using the Nemenyi test shows statistically significant differences in performance between our method and existing algorithms, with a critical distance (CD = 1.856, <mml:math altimg=\"si1.svg\" display=\"inline\"><mml:mi>p</mml:mi></mml:math>-value <mml:math altimg=\"si2.svg\" display=\"inline\"><mml:mrow><mml:mo><</mml:mo><mml:mn>0</mml:mn><mml:mo>.</mml:mo><mml:mn>05</mml:mn></mml:mrow></mml:math>), demonstrating the superiority of our approach. Experimental results on synthetic data highlight the framework’s robustness in handling noise and redundancy, demonstrating its effectiveness in challenging environments. Validation on eight real-world datasets, including electroencephalography and Alzheimer’s neuroimaging data, confirms its superior performance, particularly with limited training samples. The implementation is available at <ce:inter-ref xlink:href=\"https://github.com/archy666/CSMVIB\" xlink:type=\"simple\">https://github.com/archy666/CSMVIB</ce:inter-ref>.","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"22 1","pages":""},"PeriodicalIF":18.6,"publicationDate":"2025-01-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142974948","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-02DOI: 10.1016/j.inffus.2025.102928
Che Xu, Peng Zhu, Jiacun Wang, Giancarlo Fortino
In the era of big data, accurately diagnosing diabetes mellitus (DM) often requires fusing diverse types of information. Machine learning has emerged as a prevalent approach to achieve this. Despite its potential, clinical acceptance remains limited, primarily due to the lack of explainability in diagnostic predictions. The emergence of explainable artificial intelligence (XAI) offers a promising solution, yet both explainable and non-explainable models rely heavily on noise-free datasets. Label noise filters (LNFs) have been designed to enhance dataset quality by identifying and removing mislabeled samples, which can improve the predictive performance of diagnostic models. However, the impact of label noise on diagnostic explanations remains unexplored. To address this issue, this paper proposes an ensemble framework for LNFs that fuses information from different LNFs through three phases. In the first phase, a diverse pool of LNFs is generated. Second, the widely-used LIME (Local Interpretable Model-Agnostic Explanations) technique is employed to provide local explainability for diagnostic predictions made by black-box models. Finally, four ensemble strategies are designed to generate the final local diagnostic explanations for DM patients. The theoretical advantage of the ensemble is also demonstrated. The proposed framework is comprehensively evaluated on four DM datasets to assess its ability to mitigate the adverse impact of label noise on diagnostic explanations, compared to 24 baseline LNFs. Experimental results demonstrate that individual LNFs fail to consistently ensure the quality of diagnostic explanations, whereas the LNF ensemble based on local explanations provides a feasible solution to this challenge.
{"title":"Improving the local diagnostic explanations of diabetes mellitus with the ensemble of label noise filters","authors":"Che Xu, Peng Zhu, Jiacun Wang, Giancarlo Fortino","doi":"10.1016/j.inffus.2025.102928","DOIUrl":"https://doi.org/10.1016/j.inffus.2025.102928","url":null,"abstract":"In the era of big data, accurately diagnosing diabetes mellitus (DM) often requires fusing diverse types of information. Machine learning has emerged as a prevalent approach to achieve this. Despite its potential, clinical acceptance remains limited, primarily due to the lack of explainability in diagnostic predictions. The emergence of explainable artificial intelligence (XAI) offers a promising solution, yet both explainable and non-explainable models rely heavily on noise-free datasets. Label noise filters (LNFs) have been designed to enhance dataset quality by identifying and removing mislabeled samples, which can improve the predictive performance of diagnostic models. However, the impact of label noise on diagnostic explanations remains unexplored. To address this issue, this paper proposes an ensemble framework for LNFs that fuses information from different LNFs through three phases. In the first phase, a diverse pool of LNFs is generated. Second, the widely-used LIME (Local Interpretable Model-Agnostic Explanations) technique is employed to provide local explainability for diagnostic predictions made by black-box models. Finally, four ensemble strategies are designed to generate the final local diagnostic explanations for DM patients. The theoretical advantage of the ensemble is also demonstrated. The proposed framework is comprehensively evaluated on four DM datasets to assess its ability to mitigate the adverse impact of label noise on diagnostic explanations, compared to 24 baseline LNFs. Experimental results demonstrate that individual LNFs fail to consistently ensure the quality of diagnostic explanations, whereas the LNF ensemble based on local explanations provides a feasible solution to this challenge.","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"14 1","pages":""},"PeriodicalIF":18.6,"publicationDate":"2025-01-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142929201","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2025-01-02DOI: 10.1016/j.inffus.2024.102918
Ling Gao, Nan Sheng, Yiming Liu, Hao Xu
Multimodal sarcasm detection is a pivotal visual-linguistic task that aims to identify incongruity between the text purpose and the underlying meaning of other modal data. Existing works are dedicated to the learning of unimodal embeddings and the fusion of multimodal information. Nonetheless, they neglect the importance of topology and incongruity between multimodal information for sarcasm detection. Therefore, we propose a novel multimodal sarcasm detection network that incorporates multimodal topology capture and incongruity perception (TCIP). A text single-mode graph, a visual single-mode graph, and a visual–text heterogeneous graph are first established, where nodes contain visual elements and text elements. The association matrix of the heterogeneous graph encapsulates visual–visual associations, text–text associations, and visual–text associations. Subsequently, TCIP learns single-modal graphs and a heterogeneous graph based on graph convolutional networks to capture text topology information, visual topology information, and multimodal topology information. Furthermore, we pull together multimodal embeddings exhibiting consistent distributions and push away those with inconsistent distributions. TCIP finally feeds the fused embedding into a classifier to detect sarcasm results within visual–text pairs. Experimental results conducted on the multimodal sarcasm detection benchmarks and the multimodal science question answering dataset demonstrate the exceptional performance of TCIP.
{"title":"TCIP: Network with topology capture and incongruity perception for sarcasm detection","authors":"Ling Gao, Nan Sheng, Yiming Liu, Hao Xu","doi":"10.1016/j.inffus.2024.102918","DOIUrl":"https://doi.org/10.1016/j.inffus.2024.102918","url":null,"abstract":"Multimodal sarcasm detection is a pivotal visual-linguistic task that aims to identify incongruity between the text purpose and the underlying meaning of other modal data. Existing works are dedicated to the learning of unimodal embeddings and the fusion of multimodal information. Nonetheless, they neglect the importance of topology and incongruity between multimodal information for sarcasm detection. Therefore, we propose a novel multimodal sarcasm detection network that incorporates multimodal topology capture and incongruity perception (TCIP). A text single-mode graph, a visual single-mode graph, and a visual–text heterogeneous graph are first established, where nodes contain visual elements and text elements. The association matrix of the heterogeneous graph encapsulates visual–visual associations, text–text associations, and visual–text associations. Subsequently, TCIP learns single-modal graphs and a heterogeneous graph based on graph convolutional networks to capture text topology information, visual topology information, and multimodal topology information. Furthermore, we pull together multimodal embeddings exhibiting consistent distributions and push away those with inconsistent distributions. TCIP finally feeds the fused embedding into a classifier to detect sarcasm results within visual–text pairs. Experimental results conducted on the multimodal sarcasm detection benchmarks and the multimodal science question answering dataset demonstrate the exceptional performance of TCIP.","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"2 1","pages":""},"PeriodicalIF":18.6,"publicationDate":"2025-01-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142929315","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In many public places, the heterogeneity of pedestrians leads to diverse travel behaviors including overtaking behavior. However, according to the variety of factors such as the heterogeneous attributes of pedestrians and the alterations of surrounding environment, the previous models for simulating overtaking behavior exist the problems of behavior loss or decision imbalance. By observing that overtaking behavior can be regarded as a process consisting of multiple micro-actions, this paper proposes a micro-action-based macro-to-micro decision-making (M3DM) framework to simulate fine-grained overtaking behavior of heterogeneous pedestrians. The framework incorporates two modules: micro-action modeling (MM) and macro-to-micro decision-making (MMDM) modules. The former module constructs the mapping relationship between proposed micro-actions and multiple personality characterization, and builds the simulation model of each micro-action. While the latter module integrates the density based macro and energy consumption based micro decision into framework, which achieves a more realistic simulation of overtaking behavior. Extensive real experiments are conducted to calibrate the parameters and verify the rationality of our framework. Moreover, two different simulation cases prove the authenticity of the proposed simulation model. The results indicate that the M3DM framework can significantly enhance the simulation accuracy of pedestrian behaviors, providing valuable insights for pedestrian flow management and safety in high-density environments.
{"title":"A micro-action-based decision-making framework for simulating overtaking behaviors of heterogeneous pedestrians","authors":"Jingxuan Peng, Zhonghua Wei, Yanyan Chen, Shaofan Wang, Yongxing Li, Liang Chen, Fujiyama Taku","doi":"10.1016/j.inffus.2024.102898","DOIUrl":"https://doi.org/10.1016/j.inffus.2024.102898","url":null,"abstract":"In many public places, the heterogeneity of pedestrians leads to diverse travel behaviors including overtaking behavior. However, according to the variety of factors such as the heterogeneous attributes of pedestrians and the alterations of surrounding environment, the previous models for simulating overtaking behavior exist the problems of behavior loss or decision imbalance. By observing that overtaking behavior can be regarded as a process consisting of multiple micro-actions, this paper proposes a micro-action-based macro-to-micro decision-making (M3DM) framework to simulate fine-grained overtaking behavior of heterogeneous pedestrians. The framework incorporates two modules: micro-action modeling (MM) and macro-to-micro decision-making (MMDM) modules. The former module constructs the mapping relationship between proposed micro-actions and multiple personality characterization, and builds the simulation model of each micro-action. While the latter module integrates the density based macro and energy consumption based micro decision into framework, which achieves a more realistic simulation of overtaking behavior. Extensive real experiments are conducted to calibrate the parameters and verify the rationality of our framework. Moreover, two different simulation cases prove the authenticity of the proposed simulation model. The results indicate that the M3DM framework can significantly enhance the simulation accuracy of pedestrian behaviors, providing valuable insights for pedestrian flow management and safety in high-density environments.","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"74 1","pages":""},"PeriodicalIF":18.6,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142929204","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：