Information Fusion最新文献

Tropical cyclones (TC) exert a profound impact on cities, causing extensive damage and losses. Thus, TC Intensity Prediction is crucial for creating sustainable cities as it enables proactive measures to be taken, including evacuation planning, infrastructure reinforcement, and emergency response coordination. In this study, we propose a Deep learning-powered TC Intensity Prediction (Deep-TCP) framework. In particular, Deep-TCP contains a data constraint module for fusing data features from multiple sources and establishing a unified global representation. To capture the spatiotemporal attributes, a Spatial-Temporal Attention (ST-Attention) module is built to distill insights from environmental variables. To improve the robustness and stability of the predictions, an encoder-decoder module that utilizes the ConvGPU unit is introduced to enhance feature maps. Then, a novel feature enhancement module is built to bolster the generalization capability and solve the dependency attenuation. The results demonstrate that the Deep-TCP framework significantly outperforms various benchmarks. Additionally, it effectively predicts multiple TC categories within the 6–24 h timeframe, showing strong capability in predicting changing trends. The reliable prediction results are potentially beneficial for disaster management and urban planning, significantly enhancing urban sustainability by improving preparedness and response strategies.

Accurate Link Prediction (LP) in Social Networks (SNs) is crucial for various practical applications, such as recommendation systems and network security. However, traditional techniques often struggle to capture the intricate and multidimensional nature of these networks. This paper presents a novel approach, the Synergetic Fusion-based Graph Convolutional Networks (SFGCN), designed to enhance LP accuracy in SNs. The SFGCN model utilizes a fusion architecture that combines structural features and other attribute data through early, intermediate, and late fusion mechanisms to create improved node and edge representations. We thoroughly evaluate our SFGCN model on seven real-world datasets, encompassing citation networks, co-purchase networks, and academic publication domains. The results demonstrate its superiority over baseline GCN architectures and other selected LP methods, achieving a 6.88 % improvement in accuracy. The experiments show that our model captures the complex interactions and dependencies within SNs, providing a comprehensive understanding of their underlying dynamics. The approach mentioned can be effectively applied in the domain of SN analysis to enhance the accuracy of LP results. This method not only improves the precision of predictions but also enhances the adaptability of the model in diverse SN scenarios.

Few-shot Learning (FSL) aims to learn and distinguish new categories from a scant number of available samples, presenting a significant challenge in the realm of deep learning. Recent researchers have sought to leverage the additional semantic or linguistic information of scarce categories with a pre-trained language model to facilitate learning, thus partially alleviating the problem of insufficient supervision signals. Nonetheless, the full potential of the semantic information and pre-trained language model have been underestimated in the few-shot learning till now, resulting in limited performance enhancements. To address this, we propose a straightforward and efficacious framework for few-shot learning tasks, specifically designed to exploit the semantic information and language model. Specifically, we explicitly harness the zero-shot capability of the pre-trained language model with learnable prompts. And we directly add the visual feature with the textual feature for inference without the intricate designed fusion modules as in prior studies. Additionally, we apply the self-ensemble and distillation to further enhance performance. Extensive experiments conducted across four widely used few-shot datasets demonstrate that our simple framework achieves impressive results. Particularly noteworthy is its outstanding performance in the 1-shot learning task, surpassing the current state-of-the-art by an average of 3.3% in classification accuracy. Our code will be available at https://github.com/zhouchunpong/SimpleFewShot.

To address real world decision problems in reinforcement learning, it is common to train a policy in a simulator first for safety. Unfortunately, the sim-real gap hinders effective simulation-to-real transfer without substantial training data. However, collecting real samples of complex tasks is often impractical, and the sample inefficiency of reinforcement learning exacerbates the simulation-to-real problem, even with online interaction or data. Representation learning can improve sample efficiency while keeping generalization by projecting high-dimensional inputs into low-dimensional representations. However, whether trained independently or simultaneously with reinforcement learning, representation learning remains a separate auxiliary task, lacking task-related features and generalization for simulation-to-real transfer. This paper proposes Siamese-Q, a new representation learning method employing Siamese networks and zero-shot simulation-to-real transfer, which narrows the distance between inputs with the same semantics in the latent space with respect to Q values. This allows us to fuse task-related information into the representation and improve the generalization of the policy. Evaluation in virtual and real autonomous vehicle scenarios demonstrates substantial improvements of 19.5% and 94.2% respectively over conventional representation learning, without requiring any real-world observations or on-policy interaction, and enabling reinforcement learning policies trained in simulations transfer to reality.

In this paper, we present a new spectral analysis and a low-dimensional embedding of two aligned multimodal datasets. Our approach combines manifold learning with the Riemannian geometry of symmetric and positive-definite (SPD) matrices. Manifold learning typically includes the spectral analysis of a single kernel matrix corresponding to a single dataset or a concatenation of several datasets. Here, we use the Riemannian geometry of SPD matrices to devise an interpolation scheme for combining two kernel matrices corresponding to two, possibly multimodal, datasets. We study the way the spectra of the kernels change along geodesic paths on the manifold of SPD matrices. We show that this change enables us, in a purely unsupervised manner, to derive an informative spectral representation of the relations between the two datasets. Based on this representation, we propose a new multimodal manifold learning method. We showcase the performance of the proposed spectral representation and manifold learning method using both simulations and real-measured data from multi-sensor industrial condition monitoring and artificial olfaction. We demonstrate that the proposed method achieves superior results compared to several baselines in terms of the truncated Dirichlet energy.

Context: Recently, many studies have been proposed to address the threat posed by Android malware. However, the continuous evolution of malware poses challenges to the task of representing application features in current detection methods. Objective: This paper introduces a novel Android malware detection approach based on the source code and binary code of software by leveraging large pre-trained models with a fine-grained multimodal fusion strategy. Method: Specifically, the approach treats the source code and binary code as the programming language modality (PM) and machine language modality (MM), respectively. Then, domain-specific knowledge (sensitive API) combined with large pre-trained model is further applied to extract PM features; while the binary code is transformed into RGB images, from which MM features are extracted using a pre-trained image processing model. Furthermore, a fine-grained fusion strategy is implemented using a multi-head self-attention mechanism to effectively capture the correlations among features across different modalities and generate comprehensive features for application malware detection. Results and Conclusion: The detection performance and generalization ability of the proposed method were validated on two experimental datasets. The results demonstrate that our method can accurately distinguish malware, achieving an accuracy of 98.28% and an F1-score of 98.66%. Additionally, it performs well on unseen data, with an accuracy of 92.86% and an F1-score of 94.49%. Meanwhile, ablation experiments confirm the contributions of sensitive API knowledge and the fine-grained multimodal fusion strategy to the success of malware detection.

3D occupancy perception technology aims to observe and understand dense 3D environments for autonomous vehicles. Owing to its comprehensive perception capability, this technology is emerging as a trend in autonomous driving perception systems, and is attracting significant attention from both industry and academia. Similar to traditional bird’s-eye view (BEV) perception, 3D occupancy perception has the nature of multi-source input and the necessity for information fusion. However, the difference is that it captures vertical structures that are ignored by 2D BEV. In this survey, we review the most recent works on 3D occupancy perception, and provide in-depth analyses of methodologies with various input modalities. Specifically, we summarize general network pipelines, highlight information fusion techniques, and discuss effective network training. We evaluate and analyze the occupancy perception performance of the state-of-the-art on the most popular datasets. Furthermore, challenges and future research directions are discussed. We hope this paper will inspire the community and encourage more research work on 3D occupancy perception. A comprehensive list of studies in this survey is publicly available in an active repository that continuously collects the latest work: https://github.com/HuaiyuanXu/3D-Occupancy-Perception.

The emergence of smart sustainable cities has unveiled a wealth of data sources, each contributing to a vast array of urban applications. At the heart of managing this plethora of data is multisource information fusion (MSIF), a sophisticated approach that not only improves the quality of data collected from myriad sources, including sensors, satellites, social media, and citizen-generated content, but also aids in generating actionable insights crucial for sustainable urban management. Unlike simple data fusion, MSIF excels in harmonizing disparate data sources, effectively navigating through their variability, potential conflicts, and the challenges posed by incomplete datasets. This capability is essential for ensuring the integrity and utility of information, which supports comprehensive insights into urban systems and effective planning. This survey combines hierarchical and multi-dimensional classification to examine how MSIF integrates and analyses diverse datasets, enhancing the operational efficiency and intelligence of urban environments. It highlights the most significant challenges and opportunities presented by MSIF in smart sustainable cities, particularly how it overcomes the limitations of existing approaches in scope and coverage.

By considering social, economic, and environmental factors, MSIF offers a multidisciplinary approach that is pivotal for advancing sustainable urban development. Recognized as an essential resource for academics and practitioners, this study promotes a new wave of MSIF innovations aimed at improving the cohesion, efficiency, and sustainability of smart cities.

Risk identification in power operations is crucial for both personal safety and power production. Existing risk identification methods mainly use target detection models to identify the common risks but the scene specificity of risk occurrence. For example, not wearing a safety harness, not wearing insulated gloves, etc. Since most methods for detecting safety gears make sense only under specific scene. But the power electric work is a complex object involving many elements such as personnel, equipment and safety tools. Therefore, this paper proposes a scene understanding method that integrates visual features and spatial relationship features among scene elements. This method constructs a scenean undirected scene graph to represent the interactive relationship among the elements, extracts the interactive features by using a graph encoder-decoder convolution module, and fuse perceived high-dimensional visual features and spatial topological features for scene recognition, in order to effectively solve addressing the power operation scene understanding problem under multi-element interaction. Finally, a power inspection operation scenario was chosen as the test case. The outcome of the evaluation indicates results indicate that the proposed approach suggested in this study exhibits superior precision in scene identification and shows ademonstrates strong generalization ability.