Neural Processing Letters最新文献

In recent years, some classical graph contrastive learning(GCL) frameworks have been proposed to address the problem of sparse labeling of graph data in the real world. However, in node classification tasks, there are two obvious problems with existing GCL frameworks: first, the stochastic augmentation methods they adopt lose a lot of semantic information; second, the local–local contrasting mode selected by most frameworks ignores the global semantic information of the original graph, which limits the node classification performance of these frameworks. To address the above problems, this paper proposes a novel graph contrastive learning framework, MDGCL, which introduces two graph diffusion methods, Markov and PPR, and a deterministic–stochastic data augmentation strategy while retaining the local–local contrasting mode. Specifically, before using the two stochastic augmentation methods (FeatureDrop and EdgeDrop), MDGCL first uses two deterministic augmentation methods (Markov diffusion and PPR diffusion) to perform data augmentation on the original graph to increase the semantic information, this step ensures subsequent stochastic augmentation methods do not lose too much semantic information. Meanwhile, the diffusion matrices carried by the augmented views contain global semantic information of the original graph, allowing the framework to utilize the global semantic information while retaining the local-local contrasting mode, which further enhances the node classification performance of the framework. We conduct extensive comparative experiments on multiple benchmark datasets, and the results show that MDGCL outperforms the representative baseline frameworks on node classification tasks. Among them, compared with COSTA, MDGCL’s node classification accuracy has been improved by 1.07% and 0.41% respectively on two representative datasets, Amazon-Photo and Coauthor-CS. In addition, we also conduct ablation experiments on two datasets, Cora and CiteSeer, to verify the effectiveness of each improvement work of our framework.

In this journal, Cheng has proposed a backpropagation (BP) procedure called BPFCC for deep fully connected cascaded (FCC) neural network learning in comparison with a neuron-by-neuron (NBN) algorithm of Wilamowski and Yu. Both BPFCC and NBN are designed to implement the Levenberg-Marquardt method, which requires an efficient evaluation of the Gauss-Newton (approximate Hessian) matrix (nabla textbf{r}^textsf{T} nabla textbf{r}), the cross product of the Jacobian matrix (nabla textbf{r}) of the residual vector (textbf{r}) in nonlinear least squares sense. Here, the dominant cost is to form (nabla textbf{r}^textsf{T} nabla textbf{r}) by rank updates on each data pattern. Notably, NBN is better than BPFCC for the multiple (q~!(>!1))-output FCC-learning when q rows (per pattern) of the Jacobian matrix (nabla textbf{r}) are evaluated; however, the dominant cost (for rank updates) is common to both BPFCC and NBN. The purpose of this paper is to present a new more efficient stage-wise BP procedure (for q-output FCC-learning) that reduces the dominant cost with no rows of (nabla textbf{r}) explicitly evaluated, just as standard BP evaluates the gradient vector (nabla textbf{r}^textsf{T} textbf{r}) with no explicit evaluation of any rows of the Jacobian matrix (nabla textbf{r}).

In order to better solve the visual detection problem of manipulator grasping non-cooperative targets, we propose a method of grasp pose detection based on pixel point and feature fusion. By using the improved U2net network as the backbone for feature extraction and feature fusion of the input image, and the grasp prediction layer detects the grasp pose on each pixel. In order to adapt the U2net to grasp pose detection and improve its detection performance, we improve detection speed and control sampling depth by simplifying its network structure, while retaining some shallow features in feature fusion to enhance its feature extraction capability. We introduce depthwise separable convolution in the grasp prediction layer, further fusing the features extracted from the backbone to obtain predictive feature maps with stronger feature expressiveness. FocalLoss is selected as the loss function to solve the problem of unbalanced positive and negative samples in network training. We use the Cornell dataset for training and testing, perform pixel-level labeling on the image, and replace the labels that are not conducive to the actual grasping. This adaptation helps the dataset better suit the network training and testing while meeting the real-world grasping requirements of the manipulator. The evaluation results on image-wise and object-wise are 95.65% and 91.20% respectively, and the detection speed is 0.007 s/frame. We also used the method for actual manipulator grasping experiments. The results show that our method has improved accuracy and speed compared with previous methods, and has strong generalization ability and portability.

Meta-learning has proven to be a powerful paradigm for transferring knowledge from prior tasks to facilitate the quick learning of new tasks in automatic speech recognition. However, the differences between languages (tasks) lead to variations in task learning directions, causing the harmful competition for model’s limited resources. To address this challenge, we introduce the task-agreement multilingual meta-learning (TAMML), which adopts the gradient agreement algorithm to guide the model parameters towards a direction where tasks exhibit greater consistency. However, the computation and storage cost of TAMML grows dramatically with model’s depth increases. To address this, we further propose a simplification called TAMML-Light which only uses the output layer for gradient calculation. Experiments on three datasets demonstrate that TAMML and TAMML-Light achieve outperform meta-learning approaches, yielding superior results.Furthermore, TAMML-Light can reduce at least 80 (%) of the relative increased computation expenses compared to TAMML.

Traditional deep learning methods require a large amount of labeled data for model training, which is laborious and costly in real word. Few-shot learning (FSL) aims to recognize novel classes with only a small number of labeled samples to address these challenges. We focus on metric-based few-shot learning with improvements in both feature extraction and metric method. In our work, we propose the Pluralistic Attention Network (PANet), a novel attention-oriented framework, involving both a local encoded intra-attention(LEIA) module and a global encoded reciprocal attention(GERA) module. The LEIA is designed to capture comprehensive local feature dependencies within every single sample. The GERA concentrates on the correlation between two samples and learns the discriminability of representations obtained from the LEIA. The two modules are complementary to each other and ensure the feature information within and between images can be fully utilized. Furthermore, we also design a dual-centralization (DC) cosine similarity to eliminate the disparity of data distribution in different dimensions and enhance the metric accuracy between support and query samples. Our method is thoroughly evaluated with extensive experiments, and the results demonstrate that with the contribution of each component, our model can achieve high-performance on four widely used few-shot classification benchmarks of miniImageNet, tieredImageNet, CUB-200-2011 and CIFAR-FS.

In this paper, the exponential stability issue of stochastic impulsive neutral neural networks driven by Lévy noise is explored. By resorting to the Lyapunov-Krasovskii function that involves neutral time-delay components, the properties of the Lévy process, as well as various inequality approaches, some sufficient exponential stability criteria in non-Lipschitz cases are obtained. Besides, the achieved results depend on the time-delay, noise intensity, and impulse factor. At the end of the paper, two numerical examples with simulations are presented to demonstrate the effectiveness and feasibility of the addressed results

Sequential recommendation is the mainstream approach in the field of click-through-rate (CTR) prediction for modeling users’ behavior. This behavior implies the change of the user’s interest, and the goal of sequential recommendation is to capture this dynamic change. However, existing studies have focused on designing complex dedicated networks to capture user interests from user behavior sequences, while neglecting the use of auxiliary information. Recently, knowledge graph (KG) has gradually attracted the attention of researchers as a structured auxiliary information. Items and their attributes in the recommendation, can be mapped to knowledge triples in the KG. Therefore, the introduction of KG to recommendation can help us obtain more expressive item representations. Since KG can be considered a special type of graph, it is possible to use the graph neural network (GNN) to propagate the rich information contained in the KG into the item representation. Based on this idea, this paper proposes a recommendation method that uses KG as auxiliary information. The method first propagates the knowledge information in the KG using GNN to obtain a knowledge-rich item representation. Then the temporal features in the item sequence are extracted using a transformer for CTR prediction, namely the Knowledge Graph-Aware Deep Interest Extraction network (KGDIE). To evaluate the performance of this model, we conducted extensive experiments on two real datasets with different scenarios. The results showed that the KGDIE method could outperform several state-of-the-art baselines. The source code of our model is available at https://github.com/gylgyl123/kgdie.

The goal of this research is to design the Gudermannian neural networks (GNNs) to solve a type of two-point nonlinear singular boundary value problems (TPN-SBVPs) that arise within thermal-explosion theory. The results of these investigation are provided for different neurons (4, 12 and 20), as well as absolute error along with the time complexity. For solving the TPN-SBVPs, a genetic algorithm (GA) and sequential quadratic programming (SQP) are used to optimize the error function. The accuracy of designed GNNs is provided by using a hybrid GA–SQP combination, which is based on a comparison of obtained and actual solutions. Furthermore, statistical analysis of the data is proposed in order to establish the competence as well as effectiveness of designed and the efficacy of the designed computing framework for solving the TPN-SBVPs.

The paper addresses the fixed-/preassigned-time synchronization of stochastic memristive neural networks (MNNs) with uncertain parameters and mixed delays. Adaptive sliding mode control (ASMC) technology is mainly utilized. First, a proper sliding surface is constructed and the adaptive laws are given. Also, the synchronization control scheme is designed, which can ensure error system to realize fixed-time stability. Second, preassigned-time sliding mode control scheme is mainly provided to realize fast synchronization of MNNs. The presented theoretical methods can guarantee the error system convergence and stability for reaching and sliding mode within preassigned-time. And the synchronization criteria and explicit expression of settling time (ST) are acquired, where ST is not related with initial values and controller parameters but can be predefined perferentially. Finally, the calculation example is offered to interpret the practicability and availability of the innovations in this paper.

Continual relation extraction aims to learn new relations from a continuous stream of data while avoiding forgetting old relations. Existing methods typically use the BERT encoder to obtain semantic embeddings, ignoring the fact that the vector representations suffer from anisotropy and uneven distribution. Furthermore, the relation prototypes are usually computed by memory samples directly, resulting in the model being overly sensitive to memory samples. To solve these problems, we propose a new continual relation extraction method. Firstly, we modified the basic structure of the sample encoder to generate uniformly distributed semantic embeddings using the supervised SimCSE-BERT to obtain richer sample information. Secondly, we introduced static relation prototypes and dynamically adjust their proportion with dynamic relation prototypes to adapt to the feature space. Lastly, through experimental analysis on the widely used FewRel and TACRED datasets, the results demonstrate that the proposed method effectively enhances semantic embeddings and relation prototypes, resulting in a further alleviation of catastrophic forgetting in the model. The code will be soon released at https://github.com/SuyueW/SS-CRE.