Neural Processing Letters最新文献

This brief investigates the extended dissipativity performance of semi-discretized competitive neural networks (CNNs) with time-varying delays. Inspired by the computational efficiency and feasibility of implementing the networks, we formulate a discrete counterpart to the continuous-time CNNs. By employing an appropriate Lyapunov–Krasovskii functional (LKF) and a relaxed summation inequality, sufficient conditions ensure the extended dissipative criteria of discretized CNNs are obtained in the linear matrix inequality framework. Finally, to refine our prediction, two numerical examples are provided to demonstrate the sustainability and merits of the theoretical results.

Abstract

Deep learning models produce impressive results in any natural language processing applications when given a better learning strategy and trained with large labeled datasets. However, the annotation of massive training data is far too expensive, especially in the legal domain, due to the need for trained legal professionals. Data augmentation solves the problem of learning without labeled big data. In this paper, we employ pre-trained language models and prompt engineering to generate large-scale pseudo-labeled data for the legal overruling task using 100 data samples. We train small recurrent and convolutional deep-learning models using this data and fine-tune a few other transformer models. We then evaluate the effectiveness of the models, both with and without data augmentation, using the benchmark dataset and analyze the results. We also test the performance of these models with the state-of-the-art GPT-3 model under few-shot setting. Our experimental findings demonstrate that data augmentation results in better model performance in the legal overruling task than models trained without augmentation. Furthermore, our best-performing deep learning model trained on augmented data outperforms the few-shot GPT-3 by 18% in the F1-score. Additionally, our results highlight that the small neural networks trained with augmented data achieve outcomes comparable to those of other large language models.

In multi-label learning, label-specific feature learning can effectively avoid some ineffectual features that interfere with the classification performance of the model. However, most of the existing label-specific feature learning algorithms improve the performance of the model for classification by constraining the solution space through label correlation. The non-equilibrium of the label distribution not only leads to some spurious correlations mixed in with the calculated label correlations but also diminishes the performance of the classification model. Causal learning can improve the classification performance and robustness of the model by capturing real causal relationships from limited data. Based on this, this paper proposes a causality-driven intra-class non-equilibrium label-specific features learning, named CNSF. Firstly, the causal relationship between the labels is learned by the Peter-Clark algorithm. Secondly, the label density of all instances is calculated by the intra-class non-equilibrium method, which is used to relieve the non-equilibrium distribution of original labels. Then, the correlation of the density matrix is calculated using cosine similarity and combined with causality to construct the causal density correlation matrix, to solve the problem of spurious correlation mixed in the label correlation obtained by traditional methods. Finally, the causal density correlation matrix is used to induce label-specific feature learning. Compared with eight state-of-the-art multi-label algorithms on thirteen datasets, the experimental results prove the reasonability and effectiveness of the algorithms in this paper.

This paper characterizes the robustness of exponential stability of fuzzy inertial neural network which contains time delays or stochastic disturbance through the estimation of upper limits of perturbations. By utilizing Gronwall-Bellman lemma, stochastic analysis, Cauchy inequality, the mean value theorem of integrals, as well as the properties of integrations, the limits of both time delays and stochastic disturbances are derived in this paper which can make the disturbed system keep exponential stability. The constraints between the two types of disturbances are provided in this paper. Examples are offered to validate our results.

Convolutional neural networks and graph convolutional neural networks are two classical deep learning models that have been widely used in hyperspectral image classification tasks with remarkable achievements. However, hyperspectral image classification models based on graph convolutional neural networks using only shallow spectral or spatial features are insufficient to provide reliable similarity measures for constructing graph structures, limiting their classification performance. To address this problem, we propose a new end-to-end hyperspectral image classification model combining 3D–2D hybrid convolution and a graph attention mechanism (3D–2D-GAT). The model utilizes the collaborative work of hybrid convolutional feature extraction module and GAT module to improve classification accuracy. First, a 3D–2D hybrid convolutional network is constructed and used to quickly extract the discriminant deep spatial-spectral features of various ground objects in hyperspectral image. Then, the graph is built based on deep spatial-spectral features to enhance the feature representation ability. Finally, a network of graph attention mechanism is adopted to learn long-range spatial relationship and distinguish the intra-class variation and inter-class similarity among different samples. The experimental results on three datasets, Indian Pine, the University of Pavia and Salinas Valley show that the proposed method can achieve higher classification accuracy compared with other advanced methods.

Intelligent manufacturing process needs to adopt distributed monitoring scenario due to its massive, high-dimensional and complex data. Distributed process monitoring has been introduced into global monitoring and local monitoring to analyze the characteristic relationship between process data. However, the existing framework methods ignore or suppress the fault information and thus cannot effectively identify the local faults and the time sequence characteristics between units in the batch production system. This paper proposes a novel distributed process monitoring framework based on Girvan-Newman algorithm modular subunit partitioning and probabilistic learning model with deep neural networks. First, Girvan-Newman algorithm is used to divide the complex manufacturing system modularized to reduce the latitude of data processing. Second, variational autoencoder (VAE) is adopted to ensure the stability of local analysis, and long short-term memory is adopted to improve the VAE model to detect global multi-time scale anomalies. Finally, distributed process fault detection is carried out for each subunit in a separate and integrated manner, and the performance of the framework in distributed process monitoring is analyzed through two fault detection indicators T2 and SPE statistics. A case study of the Tennessee Eastman Process is used to demonstrate the performance and applicability of the proposed framework. Results show that the proposed VAE enhancement framework based on the DNN could accurately identify faults in distributed process monitoring and locate the specific sub-units where the fault occurs. Compared with VAE-DNN method and traditional process monitoring methods, the framework proposed in this paper has higher fault detection rate and lower false alarm rate, and the detection rate of some faults can reach 100%.

Fuzzy extreme learning machine (FELM) is an effective algorithm for dealing with classification problems with noises, which uses a membership function to effectively suppress noise in data. However, FELM has the following drawbacks: (a) The membership degree of samples in FELM is constructed by considering only the distance between the samples and the class center, not the local information of samples. It is easy to mistake some boundary samples for noises. (b) FELM uses the least squares loss function, which leads to sensitivity to feature noise and instability to re-sampling. To address the above drawbacks, we propose an intuitionistic fuzzy extreme learning machine with the truncated pinball loss (TPin-IFELM). Firstly, we use the K-nearest neighbor (KNN) method to obtain local information of the samples and then construct membership and non-membership degrees for each sample in the random mapping feature space based on valuable local information. Secondly, we calculate the score value of samples based on the membership and non-membership degrees, which can effectively identify whether the boundary samples are noises or not. Thirdly, in order to maintain the sparsity and robustness of the model, and enhance the stability of the resampling of the model, we introduce the truncated pinball loss function into the model. Finally, in order to solve more efficiently, we employ the concave-convex procedure (CCCP) to solve TPin-IFELM. Extensive comparative experiments are conducted on the benchmark datasets to verify the superior performance of TPin-IFELM.

Images once were considered a reliable source of information. However, when photo-editing software started to get noticed it gave rise to illegal activities which is called image tampering. These days we can come across innumerable tampered images across the internet. Software such as Photoshop, GNU Image Manipulation Program, etc. are applied to form tampered images from real ones in just a few minutes. To discover hidden signs of tampering in an image deep learning models are an effective tool than any other methods. Models used in deep learning are capable of extracting intricate features from an image automatically. Here we proposed a combination of traditional handcrafted features along with a deep learning model to differentiate between authentic and tampered images. We have presented a dual-branch Convolutional Neural Network in conjunction with Error Level Analysis and noise residuals from Spatial Rich Model. For our experiment, we utilized the freely accessible CASIA dataset. After training the dual-branch network for 16 epochs, it generated an accuracy of 98.55%. We have also provided a comparative analysis with other previously proposed work in the field of image forgery detection. This hybrid approach proves that deep learning models along with some well-known traditional approaches can provide better results for detecting tampered images.

Deep learning has been widely used in many applications such as face recognition, autonomous driving, etc. However, deep learning models are vulnerable to various adversarial attacks, among which backdoor attack is emerging recently. Most of the existing backdoor attacks use the same trigger or the same trigger generation approach to generate the poisoned samples in the training and testing sets, which is also commonly adopted by many backdoor defense strategies. In this paper, we develop an enhanced backdoor attack (EBA) that aims to reveal the potential flaws of existing backdoor defense methods. We use a low-intensity trigger to embed the backdoor, while a high-intensity trigger to activate it. Furthermore, we propose an enhanced coalescence backdoor attack (ECBA) where multiple low-intensity incipient triggers are designed to train the backdoor model, and then, all incipient triggers are gathered on one sample and enhanced to launch the attack. Experiment results on three popular datasets show that our proposed attacks can achieve high attack success rates while maintaining the model classification accuracy of benign samples. Meanwhile, by hiding the incipient poisoned samples and preventing them from activating the backdoor, the proposed attack exhibits significant stealth and the ability to evade mainstream defense methods during the model training phase.

Arabic Handwriting Recognition (AHR) is a complex task involving the transformation of handwritten Arabic text from image format into machine-readable data, holding immense potential across various applications. Despite its significance, AHR encounters formidable challenges due to the intricate nature of Arabic script and the diverse array of handwriting styles. In recent years, Convolutional Neural Networks (CNNs) have emerged as a pivotal and promising solution to address these challenges, demonstrating remarkable performance and offering distinct advantages. However, the dominance of CNNs in AHR lacks a dedicated comprehensive review in the existing literature. This review article aims to bridge the existing gap by providing a comprehensive analysis of CNN-based methods in AHR. It covers both segmentation and recognition tasks, delving into advancements in network architectures, databases, training strategies, and employed methods. The article offers an in-depth comparison of these methods, considering their respective strengths and limitations. The findings of this review not only contribute to the current understanding of CNN applications in AHR but also pave the way for future research directions and improved practices, thereby enriching and advancing this critical domain. The review also aims to uncover genuine challenges in the domain, providing valuable insights for researchers and practitioners.