Neural Networks最新文献

Self-attention models in sequential recommendation face two under-explored but complementary challenges: (1) susceptibility to attention noise, especially in long-sequence modeling, and (2) difficulty in distinguishing between repetition and exploration behaviors under the Softmax bottleneck. To jointly tackle these challenges, we propose a unified framework with gated differential amplified attention and repetition-exploration intent modeling (GDA-REIM for short). Existing methods generally address these problems individually, via external denoising or architecture-level branching for repetition-exploration modeling. However, noise in attention weights can obscure true user intent (repetition vs. exploration), while clear intent boundaries can guide more effective denoising, making joint optimization essential yet unexplored in prior work. GDA-REIM incorporates a gated differential amplified attention (GDAA) module, which employs a three-stage "differentiation-gating-amplification" pipeline that computes and subtracts paired attention maps to suppress common-mode noise and dynamically rescales the denoised signal. Leveraging the resulting denoised representations, a partitioned intent scoring (PIS) component together with an intent discrimination margin (IDM) loss explicitly distinguishes repetition and exploration intent. Extensive experiments on ML-1M, Amazon-Video-Games, and Twitch-100k datasets demonstrate consistent improvements over strong baselines (e.g., approximately +10% improvement in NDCG@10, or N@10 for short, on ML-1M). Our code is released at https://anonymous.4open.science/r/GDA-REIM/.

The core challenge of multi-modal object re-identification (ReID) lies in reconciling the style discrepancies across different modalities with the semantic consistency of identity. However, existing methods are difficult to effectively separate semantic features from modality-specific styles, resulting in semantic representations being contaminated by noise and affecting recognition performance. To address the above issues, we propose a multi-modal re-identification framework based on semantic-stylistic decoupled distillation, named SD²-ReID (Semantic-Stylistic Decoupled Distillation for ReID), aiming to improve modal consistency and cross-modal semantic discrimination. Firstly, we design a Hybrid Multi-modal Feature Extractor (HMFE) that employs a shared shallow structure and modality-specific deep branches to achieve fine-grained feature extraction, thereby improving learning efficiency while preserving modality-specific characteristics; secondly, we design a Decoupled Distillation Module (DDM) that explicitly separates semantic and stylistic features through dual constraints of semantic and style distillation, improving cross-modal semantic consistency and discriminative ability; finally, we propose an attention-guided masking strategy and integrate intra-modal and cross-modal contrastive learning to construct a Hierarchical Self-supervised Learning Module (HSLM), thereby enhancing the model's robustness to local occlusions and style variations.The synergistic enhancement of semantic consistency, modal invariance and feature robustness is finally realized. Unlike existing methods, SD²-ReID does not require the design of a multi-modal fusion module and does not introduce additional overhead in the inference phase, while balancing recognition performance and inference efficiency. Experiments on three multi-modal object ReID benchmark test sets fully validate the effectiveness of our method.

Research on underwater object detection has primarily focused on addressing degraded imagery. Single-frame feature refinement is inherently limited by restricted static spatial information, while joint image enhancement and detection paradigms encounter non-trivial challenges arising from irreversible artifacts and conflicting optimization objectives. In contrast, temporal information from video sequences offers a direct solution. Temporal semantic information enhances the feature representation of degraded underwater frames, whereas temporal positional cues furnish dynamic geometric associations that facilitate precise object localization. We propose Transformer Underwater Spatial-Temporal Cross-domain Collaborative Detection (TransUTD), reformulating underwater degraded feature representation as a temporal contextual modeling problem. By synergistic exploitation of spatial-temporal information, TransUTD naturally learns complementary features across frames to compensate for feature degradation in key frames, rather than relying on specific heuristic components. This simplifies the detection pipeline and eliminates hand-crafted modules. In our framework, the spatial-temporal fusion encoder aggregates multi-frame features to strengthen semantic representations in degraded images. The spatial-temporal query interaction refines localization in complex underwater scenes by correlating spatial-temporal geometric cues. Finally, the temporal hybrid collaborative decoder performs dense supervision through collaborative optimization of temporal positive queries. Concurrently, we construct UVID, the first underwater video object detection dataset. Experimental evaluations demonstrate that TransUTD achieves state-of-the-art performance, delivering AP improvements of 1.5% and 1.9% on the DUO and UVID datasets, respectively. Moreover, it attains near SOTA performance on ImageNetVID with AP₅₀ of 86.0%. Our dataset and code are available at https://github.com/Anchor1566/TransUTD.

Hashing techniques are widely adopted in large-scale retrieval due to their low time and space complexity. Existing deep cross-modal hashing methods mostly rely on mini-batch training, where only a limited number of samples are processed in each iteration, often resulting in incomplete neighborhood exploration and sub-optimal embedding learning, especially on complex multi-label datasets. To address this limitation, recent studies have optimized the SoftMax loss, which is essentially equivalent to a smoothed triplet constraint with a single center assigned to each class. However, in practical retrieval scenarios, class distributions in the embedding space often contain multiple semantic clusters. Modeling each class with only one center fails to capture these intra-class local structures, thereby widening the semantic gap between heterogeneous samples. To alleviate this issue, we propose a novel deep cross-modal hashing framework, Deep SoftTriple Hashing (DSTH), which learns compact hash codes to better preserve semantic similarities in the embedding space. The framework introduces multiple centers for each class to effectively model the implicit distribution of heterogeneous samples and reduce intra-class semantic variance. To determine the number of centers, a class-center strategy is developed, where similar centers are encouraged to aggregate through an L_2,1 regularization to obtain a compact set of centers. In addition, a semantic position quantization loss is introduced to minimize quantization error and enhance the discriminability of binary codes. Extensive experiments on three multi-label datasets demonstrate the effectiveness of DSTH in cross-modal retrieval, achieving absolute mAP improvements of 1.2%-9.3% over strong baselines while consistently maintaining superior performance on PR curves. The source code is available at: https://github.com/QinLab-WFU/DSTH-SoftTriple.

Multi-atlas brain networks offer a more comprehensive and intricate understanding than a single atlas in identifying brain disorders. Traditional multi-atlas analysis methods depend on some simple fusion methods (i.e., addition and concatenation) but do not consider the information redundancy caused by increased brain regions and uncertain information between multiple atlases. To address this, we propose an effective multi-atlas brain network analysis method with a sparse and uncertain mechanism, called Sparse and Uncertain Fusion Neural Network (SUFNN). We first construct a multi-atlas brain network based on functional magnetic resonance imaging (fMRI) using different atlases. Then, an attention-enhanced module is used to learn the features of each atlas. These features are fed into the multi-atlas brain regions selection module, which can select disease-related brain regions based on their importance scores. Subsequently, the model employs the selected features for downstream processing. Finally, we employ an uncertain fusion module that determines the uncertainty of each atlas and performs an uncertain fusion strategy to get the results at the evidence level. Experimental results on the SRPBS dataset demonstrate that our SUFNN outperforms several state-of-the-art methods in identifying brain disorders.

Existing unsupervised anomaly detection methods for multivariate time series(MTS) have demonstrated advanced performance on numerous public datasets. However, these methods exhibit two critical limitations: (1) the assumption of completely noise-free training data contradicts real-world conditions where normal samples are inevitably contaminated; (2) the inherent non-stationarity of MTS induces distribution shift, leading to biased learning and degraded generalization capabilities. This paper proposes NORDA, a novel MTS anomaly detection framework that integrates a multi-order difference mechanism with distribution shift optimization. Firstly, a multi-order difference mechanism performs multi-order explicit differencing on raw temporal signals, effectively mitigating noise interference during representation learning. Secondly, a mixed reversible normalization module is proposed, employing a normalization network with multiple statistical features to dynamically model non-stationary variations across variables. This module achieves remove and restore of non-stationary properties of MTS through a symmetric reversible architecture, thereby enhancing the dynamic adaptability of model to distribution shift. By synergistically integrating the two aforementioned modules with a Transformer-based multi-layer encoder, this framework can extract robust latent representations through modeling of inter-channel dependencies in differentially processed data streams. Extensive experiments on seven benchmark datasets demonstrate that NORDA significantly outperforms sixteen typical baseline methods while exhibiting great robustness against noise contamination.

Long-term time series forecasting (LTSF) is critical to industrial applications. While recent advances mainly focus on modeling complex multivariate interactions, the practical benefits may only be marginal by challenges such as asynchronous data drifts, noise interference, and abrupt changes. This study demonstrates a well-designed univariate modeling can be more effective and efficient. The univariate model with Power Decomposition and online Post-Calibration (PDCNet) is proposed, which incorporates two novel mechanisms. 1) Power decomposition (P3D) is designed to disentangle time series based on data power distribution, which significantly enhancing data predictability and mitigating the obscuring effect from dominant periodicities. Predictability-ACF joint analysis is introduced to determine optimal decomposition thresholds. 2) By designing the abrupt factor M to classify the data morphological changes and historical performance-based correction dictionary, a lightweight online Post-Calibration is proposed to adapt to pattern drifts without retraining the main model. Comprehensive experiments show that PDCNet consistently outperforms state-of-the-art models on univariate tasks. Through simple aggregation, it also achieves top-tier multivariate performance. P3D brings an average 15% improvement in 68.57% of cases, while calibration further improves accuracy by 16% in calibrated regions. Notably, PDCNet reduces GPU memory usage which can reach up to 95% compared to multivariate counterparts in extreme-scale tasks. Our work proves that capturing internal temporal dependencies within each variable is a more efficient and practical design for LTSF. PDCNet can serve as a competitive baseline, offering superior performance with significantly reduced memory footprint. The related source code and configuration files can be accessed at https://github.com/hongchichen/PDCNet.

Multi-view graph clustering, a fundamental task in data mining and machine learning, aims to partition nodes into disjoint groups by leveraging complementary information from multiple data sources. Although significant progress has been made, existing methods often struggle to effectively capture both the unique structural information within each view and the complementary relationships across different views. Moreover, the lack of mechanisms to enforce global semantic consistency frequently results in unstable consensus representations and degraded clustering quality. To address these issues, we propose a novel end-to-end method, Multi-view Graph Clustering via Dual attention fusion and Collaborative optimization (MGCDC). Specifically, each view is first encoded using a graph attention autoencoder to obtain view-specific node embeddings. These embeddings are then integrated via a view-level attention mechanism to generate a unified consensus representation. To guide the learning process, we introduce two collaborative optimization objectives. First, a cross-view cluster alignment loss is employed to jointly perform self-training learning on both the view-specific and consensus embeddings. Second, a semantic consistency enhancement loss is introduced to maximize mutual information between node embeddings and their corresponding cluster summaries. The entire model is optimized end-to-end by jointly learning node representations, integrating multi-view information, and refining cluster assignments. Extensive experiments on five benchmark datasets demonstrate that MGCDC achieves highly competitive performance compared to state-of-the-art methods.