Neural Networks最新文献

Learning on hypergraphs has garnered significant attention recently due to their ability to effectively represent complex higher-order interactions among multiple entities compared to conventional graphs. Nevertheless, the majority of existing methods are direct extensions of graph neural networks, and they exhibit noteworthy limitations. Specifically, most of these approaches primarily rely on either the Laplacian matrix with information distortion or heuristic message passing techniques. The former tends to escalate algorithmic complexity, while the latter lacks a solid theoretical foundation. To address these limitations, we propose a novel hypergraph neural network named I²HGNN, which is grounded in an energy minimization function formulated for hypergraphs. Our analysis reveals that propagation layers align well with the message-passing paradigm in the context of hypergraphs. I²HGNN achieves a favorable trade-off between performance and interpretability. Furthermore, it effectively balances the significance of node features and hypergraph topology across a diverse range of datasets. We conducted extensive experiments on 15 datasets, and the results highlight the superior performance of I²HGNN in the task of hypergraph node classification across nearly all benchmarking datasets.

Cortical networks are capable of unsupervised learning and spontaneous replay of complex temporal sequences. Endowing artificial spiking neural networks with similar learning abilities remains a challenge. In particular, it is unresolved how different plasticity rules can contribute to both learning and the maintenance of network stability during learning. Here we introduce a biologically inspired form of GABA-Modulated Spike Timing-Dependent Plasticity (GMS) and demonstrate its ability to permit stable learning of complex temporal sequences including natural language in recurrent spiking neural networks. Motivated by biological findings, GMS utilizes the momentary level of inhibition onto excitatory cells to adjust both the magnitude and sign of Spike Timing-Dependent Plasticity (STDP) of connections between excitatory cells. In particular, high levels of inhibition in the network cause depression of excitatory-to-excitatory connections. We demonstrate the effectiveness of this mechanism during several sequence learning experiments with character- and token-based text inputs as well as visual input sequences. We show that GMS maintains stability during learning and spontaneous replay and permits the network to form a clustered hierarchical representation of its input sequences. Overall, we provide a biologically inspired model of unsupervised learning of complex sequences in recurrent spiking neural networks.

Federated learning (FL) enables a decentralized machine learning paradigm for multiple clients to collaboratively train a generalized global model without sharing their private data. Most existing works have focused on designing FL systems for unimodal data, limiting their potential to exploit valuable multimodal data for future personalized applications. Moreover, the majority of FL approaches still rely on labeled data at the client side, which is often constrained by the inability of users to self-annotate their data in real-world applications. In light of these limitations, we propose a novel multimodal FL framework, namely FedMEKT, based on a semi-supervised learning approach to leverage representations from different modalities. To address the challenges of modality discrepancy and labeled data constraints in existing FL systems, our proposed FedMEKT framework comprises local multimodal autoencoder learning, generalized multimodal autoencoder construction, and generalized classifier learning. Bringing this concept into the proposed framework, we develop a distillation-based multimodal embedding knowledge transfer mechanism which allows the server and clients to exchange joint multimodal embedding knowledge extracted from a multimodal proxy dataset. Specifically, our FedMEKT iteratively updates the generalized global encoders with joint multimodal embedding knowledge from participating clients through upstream and downstream multimodal embedding knowledge transfer for local learning. Through extensive experiments on four multimodal datasets, we demonstrate that FedMEKT not only achieves superior global encoder performance in linear evaluation but also guarantees user privacy for personal data and model parameters while demanding less communication cost than other baselines.

Recurrent neural networks (RNNs) are an important class of models for learning sequential behavior. However, training RNNs to learn long-term dependencies is a tremendously difficult task, and this difficulty is widely attributed to the vanishing and exploding gradient (VEG) problem. Since it was first characterized 30 years ago, the belief that if VEG occurs during optimization then RNNs learn long-term dependencies poorly has become a central tenet in the RNN literature and has been steadily cited as motivation for a wide variety of research advancements. In this work, we revisit and interrogate this belief using a large factorial experiment where more than 40,000 RNNs were trained, and provide evidence contradicting this belief. Motivated by these findings, we re-examine the original discussion that analyzed latching behavior in RNNs by way of hyperbolic attractors, and ultimately demonstrate that these dynamics do not fully capture the learned characteristics of RNNs. Our findings suggest that these models are fully capable of learning dynamics that do not correspond to hyperbolic attractors, and that the choice of hyper-parameters, namely learning rate, has a substantial impact on the likelihood of whether an RNN will be able to learn long-term dependencies.

The weight-sharing mechanism of convolutional kernels ensures the translation equivariance of convolutional neural networks (CNNs) but not scale and rotation equivariance. This study proposes a SIM(2) Lie group-CNN, which can simultaneously keep scale, rotation, and translation equivariance for image classification tasks. The SIM(2) Lie group-CNN includes a lifting module, a series of group convolution modules, a global pooling layer, and a classification layer. The lifting module transfers the input image from Euclidean space to Lie group space, and the group convolution is parameterized through a fully connected network using the Lie Algebra coefficients of Lie group elements as inputs to achieve scale and rotation equivariance. It is worth noting that the mapping relationship between SIM(2) and its Lie Algebra and the distance measure of SIM(2) are defined explicitly in this paper, thus solving the problem of the metric of features on the space of SIM(2) Lie group, which contrasts with other Lie groups characterized by a single element, such as SO(2). The scale-rotation equivariance of Lie group-CNN is verified, and the best recognition accuracy is achieved on three categories of image datasets. Consequently, the SIM(2) Lie group-CNN can successfully extract geometric features and perform equivariant recognition on images with rotation and scale transformations.

Current physics-informed neural network (PINN) implementations with sequential learning strategies often experience some weaknesses, such as the failure to reproduce the previous training results when using a single network, the difficulty to strictly ensure continuity and smoothness at the time interval nodes when using multiple networks, and the increase in complexity and computational overhead. To overcome these shortcomings, we first investigate the extrapolation capability of the PINN method for time-dependent PDEs. Taking advantage of this extrapolation property, we generalize the training result obtained in a specific time subinterval to larger intervals by adding a correction term to the network parameters of the subinterval. The correction term is determined by further training with the sample points in the added subinterval. Secondly, by designing an extrapolation control function with special characteristics and combining it with a correction term, we construct a new neural network architecture whose network parameters are coupled with the time variable, which we call the extrapolation-driven network architecture. Based on this architecture, using a single neural network, we can obtain the overall PINN solution of the whole domain with the following two characteristics: (1) it completely inherits the local solution of the interval obtained from the previous training, (2) at the interval node, it strictly maintains the continuity and smoothness that the true solution has. The extrapolation-driven network architecture allows us to divide a large time domain into multiple subintervals and solve the time-dependent PDEs one by one in a chronological order. This training scheme respects the causality principle and effectively overcomes the difficulties of the conventional PINN method in solving the evolution equation on a large time domain. Numerical experiments verify the performance of our method. The data and code accompanying this paper are available at https://github.com/wangyong1301108/E-DNN.

This article is concerned with the deterministic finite automaton-mode-dependent (DFAMD) exponential stability problem of impulsive switched memristive neural networks (SMNNs) with aperiodic asynchronous attacks and the network covert channel. First, unlike the existing literature on SMNNs, this article focuses on DFA to drive mode switching, which facilitates precise system behavior modeling based on deterministic rules and input characters. To eliminate the periodicity and consistency constraints of traditional attacks, this article presents the multichannel aperiodic asynchronous denial-of-service (DoS) attacks, allowing for the diversity of attack sequences. Meanwhile, the network covert channel with a security layer is exploited and its dynamic adjustment is realized jointly through the dynamic weighted try-once-discard (DWTOD) protocol and selector, which can reduce network congestion, improve data security, and enhance system defense capability. In addition, this article proposes a novel mode-dependent hybrid controller composed of output feedback control and mode-dependent impulsive control, with the goal of increasing system flexibility and efficiency. Inspired by the semi-tensor product (STP) technique, Lyapunov-Krasovskii functions, and inequality technology, the novel exponential stability conditions are derived. Finally, a numerical simulation is provided to illustrate the effectiveness of the developed approach.

In practice, collecting auxiliary labeled data with same feature space from multiple domains is difficult. Thus, we focus on the heterogeneous transfer learning to address the problem of insufficient sample sizes in neuroimaging. Viewing subjects, time, and features as dimensions, brain activation and dynamic functional connectivity data can be treated as high-order heterogeneous data with heterogeneity arising from distinct feature space. To use the heterogeneous priori knowledge from the low-dimensional brain activation data to improve the classification performance of high-dimensional dynamic functional connectivity data, we propose a tensor dictionary-based heterogeneous transfer learning framework. It combines supervised tensor dictionary learning with heterogeneous transfer learning for enhance high-order heterogeneous knowledge sharing. The former can encode the underlying discriminative features in high-order data into dictionaries, while the latter can transfer heterogeneous knowledge encoded in dictionaries through feature transformation derived from mathematical relationship between domains. The primary focus of this paper is gender classification using fMRI data to identify emotion-related brain gender differences during adolescence. Additionally, experiments on simulated data and EEG data are included to demonstrate the generalizability of the proposed method. Experimental results indicate that incorporating prior knowledge significantly enhances classification performance. Further analysis of brain gender differences suggests that temporal variability in brain activity explains differences in emotion regulation strategies between genders. By adopting the heterogeneous knowledge sharing strategy, the proposed framework can capture the multifaceted characteristics of the brain, improve the generalization of the model, and reduce training costs. Understanding the gender specific neural mechanisms of emotional cognition helps to develop the gender-specific treatments for neurological diseases.

Heterogeneous Information Networks (HINs) play a crucial role in modeling complex social systems, where predicting missing links/relations is a significant task. Existing methods primarily focus on pairwise relations, but real-world scenarios often involve multi-entity interactions. For example, in academic collaboration networks, an interaction occurs between a paper, a conference, and multiple authors. These higher-order relations are prevalent but have been underexplored. Moreover, existing methods often neglect the causal relationship between the global graph structure and the state of relations, limiting their ability to capture the fundamental factors driving relation prediction. In this paper, we propose HINCHOR, an end-to-end model for higher-order relation prediction in HINs. HINCHOR introduces a higher-order structure encoder to capture multi-entity proximity information. Then, it focuses on a counterfactual question: "If the global graph structure were different, would the higher-order relation change?" By presenting a counterfactual data augmentation module, HINCHOR utilizes global structure information to generate counterfactual relations. Through counterfactual learning, HINCHOR estimates causal effects while predicting higher-order relations. The experimental results on four constructed benchmark datasets show that HINCHOR outperforms existing state-of-the-art methods.

Interactive segmentation allows active user participation to enhance output quality and resolve ambiguities. This may be especially indispensable to medical image segmentation to address complex anatomy and customization to varying user requirements. Existing approaches often encounter issues such as information dilution, limited adaptability to diverse user interactions, and insufficient response. To address these challenges, we present a novel 3D interactive framework P²ED that divides the task into four quadrants. It is equipped with a multi-granular prompt encrypted to extract prompt features from various hierarchical levels, along with a progressive hierarchical prompt decrypter to adaptively heighten the attention to the scarce prompt features along three spatial axes. Finally, it is appended by a calibration module to further align the prediction with user intentions. Extensive experiments demonstrate that the proposed P²ED achieves accurate results with fewer user interactions compared to state-of-the-art methods and is effective in promoting the upper limit of segmentation performance. The code will be released in https://github.com/chuyhu/P2ED.