Neural Networks最新文献

Existing graph knowledge distillation methods suffer from limited absorption of the teacher's "dark knowledge" because they rely on simple logit alignment, which often causes overfitting or incomplete capture of underlying patterns. Additionally, relying on a single perspective severely restricts the student's learning effectiveness and generalization ability. To address these issues, we develop a novel Multiple Interpretation Ensemble Distillation (MIED) method. It constructs a multi-interpreter composed of multiple single-layer MLPs for the student, termed the Student Interpretation (SI) component, to interpret knowledge from diversified outputs, thus avoiding representational bias from a single student output. Based on this, it introduces two effective strategies, i.e., Hybrid Sampling and Hierarchical Update. The former employs different sampling strategies for the outputs of the teacher and student (including the SI component). Specifically, the teacher's output adopts a percentage random sampler, while the outputs of the student and SI component both leverage a positive-negative sampler. With this design, MIED can facilitate better coordination of sample selection and the learning process among the teacher, student, and SI component. The latter updates the parameters of the last layer in the student using the exponential moving average of the fused parameters of the SI component, while the parameters of other layers are updated via a regular optimizer. This enhances the robustness and generalization performance of MIED. Extensive experiments on seven real-world public datasets demonstrate that MIED outperforms existing methods in node classification tasks, resulting in an average improvement of 5.56% over GCN and 27.43% over MLP, respectively. Moreover, compared with directly using multiple students (where the number is consistent with the number of layers in the SI component), MIED achieves improvements approximately 6.00% in time, 50.00% in space, and 0.20% in accuracy. These results indicate that MIED is scalable and generalizable, and exhibits robustness on complex samples.

With the advancement of high-precision remote sensing equipment and precision measurement technology, object detection based on remote sensing images (RSIs) has been widely used in military and civilian fields. Different from traditional general-purpose environments, remote sensing presents unique challenges that significantly complicate the detection process. Specifically: (1) RSIs cover extensive monitoring areas, resulting in complex and textured backgrounds; and (2) objects often exhibit cluttered distributions, small sizes, and considerable scale variations across categories. To effectively address these challenges, we propose a Multi-Scale Pattern-Aware Task-Gating Network (MPTNet) for remote sensing object detection. First, we design a Multi-Scale Pattern-Aware Network (MPNet) backbone that employs a small and large kernel convolutional complementary strategy to capture both large-scale and small-scale spatial patterns, yielding more comprehensive semantic features. Next, we introduce a Multi-Head Cross-Space Encoder (MCE) that improves semantic fusion and spatial representation across hierarchical levels. By combining a multi-head mechanism with directional one-dimensional strip convolutions, MCE enhances spatial sensitivity at the pixel level, thus improving object localization in densely textured scenes. To harmonize cross-task synergy, we propose a Dynamic Task-Gating (DTG) head that adaptively recalibrates spatial feature representations between classification and localization branches. Extensive experimental validations on three publicly available datasets, including VisDrone, DIOR, and COCO-mini, demonstrate that our method achieves excellent performance, obtaining AP₅₀ scores of 43.3%, 80.6%, and 49.5%, respectively.

We propose Gradient-Informed Neural Networks (gradinn s), a methodology that can be used to efficiently approximate a wide range of functions in low-data regimes, when only general prior beliefs are available, a condition that is often encountered in complex engineering problems. gradinn s incorporate prior beliefs about the first-order derivatives of the target function to constrain the behavior of its gradient, thus implicitly shaping it, without requiring explicit access to the target function's derivatives. This is achieved by using two Neural Networks: one modeling the target function and a second, auxiliary network expressing the prior beliefs about the first-order derivatives (e.g., smoothness, oscillations, etc.). A customized loss function enables the training of the first network while enforcing gradient constraints derived from the auxiliary network; at the same time, it allows these constraints to be relaxed in accordance with the training data. Numerical experiments demonstrate the advantages of gradinn s, particularly in low-data regimes, with results showing strong performance compared to standard Neural Networks across the tested scenarios, including synthetic benchmark functions and real-world engineering tasks.

Graph-level clustering aims to group graphs into distinct clusters based on shared structural characteristics or semantic similarities. However, existing graph-level clustering methods generally assume that the input graph structure is complete and overlook the problem of missing relationships that commonly exist in real-world scenarios. These unmodeled missing relationships will lead to the accumulation of structural information distortion during the graph representation learning process, significantly reducing the clustering performance. To this end, we propose a novel method, Structure-Missing Graph-Level Clustering Network (SMGCN), which includes a structure augmentation module LR-SEA, an Anchor Positioning Mechanism, and Joint Contrastive Optimization. Specifically, we first output augmented graphs based on low-rank matrix completion, perform cluster matching using the Hungarian algorithm to obtain anchors, and then force same clustering graphs to converge to the corresponding anchors in the embedding space. According to our research, this is the first time that the graph-level clustering task with missing relations is proposed, and the superiority of our method is demonstrated through experiments on five benchmark datasets, compared with the state-of-the-art methods. Our source codes are available at https://github.com/MrHuSN/SMGCN.

Emotion recognition brain-computer interface (BCI) using electroencephalography (EEG) is crucial for human-computer interaction, medicine, and neuroscience. However, the scarcity of labeled EEG data limits progress in this field. To address this, self-supervised learning has gained attention as a promising approach. Despite its potential, self-supervised methods face two key challenges: (1) ensuring emotion-related information is effectively preserved, as its loss can degrade emotion recognition performance, and (2) overcoming inter-subject variability in EEG signals, which hinders generalization across subjects. To tackle these issues, we propose a novel knowledge-driven self-supervised learning framework for EEG emotion recognition. Our method incorporates domain knowledge to approximate the extraction of statistical feature differential entropy (DE), aiming to preserve emotion-related and generalizable information. The framework consists of two cascaded components as hard and soft alignments: a multi-branch convolutional differential entropy learning (MCDEL) module that simulates the DE extraction process, and a contrastive entropy alignment (CEA) module that exposes complex emotional semantics in high-dimensional space. Experiment results show that our method exhibits superior performance over existing self-supervised methods. The subject-independent mean accuracy and standard deviation of our method reached 84.48% ± 5.79 on SEED and 67.64% ± 6.35 and 68.63% ± 7.77 on the Arousal and Valence dimensions of DREAMER, respectively. We conduct an ablation study to demonstrate the contribution of each proposed component. Moreover, the t-SNE visualization intuitively presents the effect of our method on reducing inter-subject variability and discriminating emotional states.

Existing value-based online reinforcement learning (RL) algorithms suffer from slow policy exploitation due to ineffective exploration and delayed policy updates. To address these challenges, we propose an algorithm called Instant Retrospect Action (IRA). Specifically, we propose Q-Representation Discrepancy Evolution (RDE) to facilitate Q-network representation learning, enabling discriminative representations for neighboring state-action pairs. In addition, we adopt an explicit method to policy constraints by enabling Greedy Action Guidance (GAG). This is achieved through backtracking historical actions, which effectively enhances the policy update process. Our proposed method relies on providing the learning algorithm with accurate k-nearest-neighbor action value estimates and learning to design a fast-adaptable policy through policy constraints. We further propose the Instant Policy Update (IPU) mechanism, which enhances policy exploitation by systematically increasing the frequency of policy updates. We further discover that the early-stage training conservatism of the IRA method can alleviate the overestimation bias problem in value-based RL. Experimental results show that IRA can significantly improve the learning efficiency and final performance of online RL algorithms on eight MuJoCo continuous control tasks.

Six-rotor unmanned aerial vehicles (UAVs) offer significant potential, but still encounter persistent challenges in achieving efficient allocation of limited resources in dynamic and complex environments. Consequently, this paper explores the prescribed-time observer-based optimal consensus control problem for six-rotor UAVs with unified prescribed performance. A practical prescribed-time optimal control scheme is constructed through embedding the prescribed-time control method with a simplified reinforcement learning framework to realize the efficient resource allocation. Leveraging a prescribed-time adjustment function, the novel updating laws for actor and critic neural networks are developed, which guarantee that six-rotor UAVs reach a desired steady state within prescribed time. Moreover, an improved distributed prescribed-time observer is established, ensuring that each follower is able to precisely estimate the velocity and position information of the leader within prescribed time. Then, a series of nonlinear transformations and mappings is proposed, which cannot only satisfy diverse performance requirements under a unified control framework through only adjusting the design parameters a priori but also improve the user-friendliness of implementation and control design. Significantly, the global performance requirement simplifies verification process of initial constraints in traditional performance control methods. Furthermore, an adaptive prescribed-time filter is introduced to address the complexity explosion issue of the backstepping method on six-rotor UAVs, while ensuring the filter error converges within prescribed time. Eventually, simulation results confirm the effectiveness of the designed method.

Time series forecasting is widely applied in fields such as energy and network security. Various prediction models based on Transformer and MLP architectures have been proposed. However, their performance may decline to varying degrees when applied to real-world sequences with significant non-stationarity. Traditional approaches generally adopt either stabilization or a combination of stabilization and non-stationarity compensation for prediction tasks. However, non-stationarity is a crucial attribute of time series; the former approach tends to eliminate useful non-stationary patterns, while the latter may inadequately capture non-stationary information. Therefore, we propose DiffMixer, which analyzes and predicts different frequencies in non-stationary time series. We use Variational Mode Decomposition (VMD) to obtain multiple frequency components of the sequence, Multi-scale Decomposition (MsD) to optimize the decomposition of downsampled sequences, and Improved Star Aggregate-Redistribute (iSTAR) to capture interdependencies between different frequency components. Additionally, we employ the Frequency domain Processing Block (FPB) to capture global features of different frequency components in the frequency domain, and Dual Dimension Fusion (DuDF) to fuse different frequency components in two dimensions, enhancing the predictive fit for various frequencies. Compared to previous state-of-the-art methods, DiffMixer reduces the Mean Squared Error (MSE), Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Symmetric Mean Absolute Percentage Error (SMAPE) by 24.5%, 12.3%, 13.5%, and 6.1%, respectively.

Incomplete multi-view clustering (IMVC) has become an area of increasing focus due to the frequent occurrence of missing views in real-world multi-view datasets. Traditional methods often address this by attempting to recover the missing views before clustering. However, these methods face two main limitations: (1) inadequate modeling of cross-view consistency, which weakens the relationships between views, especially with a high missing rate, and (2) limited capacity to generate realistic and diverse missing views, leading to suboptimal clustering results. To tackle these issues, we propose a novel framework, Joint Generative Adversarial Network and Alignment Adversarial (JGA-IMVC). Our framework leverages adversarial learning to simultaneously generate missing views and enforce consistency alignment across views, ensuring effective reconstruction of incomplete data while preserving underlying structural relationships. Extensive experiments on benchmark datasets with varying missing rates demonstrate that JGA-IMVC consistently outperforms current state-of-the-art methods. The model achieves improvements of 3 % to 5 % in key clustering metrics such as Accuracy, Normalized Mutual Information (NMI), and Adjusted Rand Index (ARI). JGA-IMVC excels under high missing conditions, confirming its robustness and generalization capabilities, providing a practical solution for incomplete multi-view clustering scenarios.

Brain-inspired neural networks, drawing insights from biological neural systems, have emerged as a promising paradigm for temporal information processing due to their inherent neural dynamics. Spiking Neural Networks (SNNs) have gained extensive attention among existing brain-inspired neural models. However, they often struggle with capturing multi-timescale temporal features due to the static parameters across time steps and the low-precision spike activities. To this end, we propose a dynamic SNN with enhanced dendritic heterogeneity to enhance the multi-timescale feature extraction capability. We design a Leaky Integrate Modulation neuron model with Dendritic Heterogeneity (DH-LIM) that replaces traditional spike activities with a continuous modulation mechanism for preserving the nonlinear behaviors while enhancing the feature expression capability. We also introduce an Adaptive Dendritic Plasticity (ADP) mechanism that dynamically adjusts dendritic timing factors based on the frequency domain information of input signals, enabling the model to capture both rapid- and slow-changing temporal patterns. Extensive experiments on multiple datasets with rich temporal features demonstrate that our proposed method achieves excellent performance in processing complex temporal signals. These optimizations provide fresh solutions for optimizing the multi-timescale feature extraction capability of SNNs, showcasing its broad application potential.