Neurocomputing最新文献

The classification of data with outliers and noise has always been one of the principal challenges within machine learning. The previous unicentric-based fuzzy twin support vector machines (SVMs) typically allot the membership through proximity to the center of the samples, which neglects the global structural information and the local neighborhood information and potentially causes confusion between fringe support vectors and outliers. In this paper, a polycentric intuitionistic fuzzy weighted least squares twin SVMs (PIFW-LSTSVM) is presented to alleviate the above issue. Concretely, the PIFW-LSTSVM model simultaneously assigns membership and nonmembership to each sample, where the membership is determined by the sample proximity to the corresponding nearest center, and nonmembership is identified by neighborhood entropy. Benefiting from the novel polycentric weighting strategy, the PIFW-LSTSVM model mitigates the impact of outliers and noise and reduces the confusion between fringe support vectors and outliers or noise, thereby boosting the generalization ability. The experiments, conducted on both artificial and real-world benchmark datasets, comprehensively demonstrate the effectiveness and superiority of the PIFW-LSTSVM model compared to other state-of-the-art models.

Self-supervised models have demonstrated remarkable performance in speech processing by learning latent representations from large amounts of unlabeled data. Adapting these models to low-resource languages yields promising results, but the computational cost of fine-tuning all model parameters is prohibitively high. Adapters offer a solution by introducing lightweight bottleneck structures into pre-trained models for downstream tasks, enabling efficient parameter adaptation. However, randomly initialized adapters often underperform in extremely low-resource scenarios. To address this issue, we explore the Meta-Adapter for self-supervised models and analyzed some limitations of Meta-Adapter including poor learning in language-specific knowledge and meta-overfitting problems. To relieve these problems, we propose the Meta-Adaptable-Adapter (MAA), a new meta leaning algorithm that adapts to low-resource languages quickly and effectively. MAA learns task-specific adapters for feature extraction, and task-independent adapters for feature combination. The experiments on three datasets show superior performance on 31 low-resource languages across seven different language families compared to other adapters, showing better generalization and extensibility.

Hyperparameter optimization of spiking neural networks (SNNs) is a difficult task which has not yet been deeply investigated in the literature. In this work, we designed a scalable constrained Bayesian based optimization algorithm that prevents sampling in non-spiking areas of an efficient high dimensional search space. These search spaces contain infeasible solutions that output no or only a few spikes during the training or testing phases, we call such a mode a “silent network”. Finding them is difficult, as many hyperparameters are highly correlated to the architecture and to the dataset. We leverage silent networks by designing a spike-based early stopping criterion to accelerate the optimization process of SNNs trained by spike timing dependent plasticity and surrogate gradient. We parallelized the optimization algorithm asynchronously, and ran large-scale experiments on heterogeneous multi-GPU Petascale architecture. Results show that by considering silent networks, we can design more flexible high-dimensional search spaces while maintaining a good efficacy. The optimization algorithm was able to focus on networks with high performances by preventing costly and worthless computation of silent networks.

Recently, fully-inductive link prediction in knowledge graphs (KGs) has aimed to predict missing links between unseen–unseen entities, independently completing evolving KGs. The latest literature emphasizes path-based graph neural network (GNN) methods, which combine traditional path-based methods with popular GNN methods, possessing generalization capability, interpretability, scalability, and high model capacity under the inductive setting. This paper presents the first comparative analysis of fully-inductive link prediction using path-based GNNs. First, we comprehensively review and summarize the research of six relevant models, divided into relational digraph-based models and Bellman–Ford algorithm-based models. Based on this, we conduct a comprehensive analysis of these models in terms of effectiveness and efficiency (including runtime, memory, and learning curves), and compared them with two subgraph-based models. Furthermore, we delve into the impact of factors such as message functions, aggregation functions, and negative sampling in the loss function on path-based GNNs. Finally, we provide an outlook on future research directions.

In this paper, we explore reconstructing high-quality clothed 3D humans from a single RGB-D image, assuming that virtual humans can be represented by front-view and back-view depths. Due to the scarcity of captured real RGB-D human images, we employ rendered images to train our method. However, rendered images lack background with significant depth variation in silhouettes, leading to shape prediction inaccuracies and noise. To mitigate this issue, we introduce a pseudo-multi-task framework, which incorporates a Conditional Generative Adversarial Network (CGAN) to infer back-view RGB-D images and a self-supervised Masked Autoencoder (MAE) to capture latent structural information of the human body. Additionally, we propose a Multi-scale Feature Fusion (MFF) module to effectively merge structural information and conditional features at various scales. Our method surpasses many existing techniques, as demonstrated through evaluations on the Thuman, RenderPeople, and BUFF datasets. Notably, our approach excels in reconstructing high-quality human models, even under challenging conditions such as complex poses and loose clothing, both on rendered and real-world images. Codes are available at https://github.com/Archaic-Atom/MaskRecon.

Inductive relation prediction is an important learning task for knowledge graph completion that aims to infer new facts from existing ones. Previous works that focus on path-based are naturally limited in expressive. The methods based on graph neural network framework consider all paths thus improving the performance. However, fusing all paths information may extract features that are spuriously correlated with the prediction. By analogy to the human reasoning process, we observe that only a small subset of the critical paths determine the prediction. In this work, we propose a novel framework that extracts such critical paths to make inductive relation prediction on Knowledge Graph with Graph Information Bottleneck (KG-GIB). KG-GIB is the first attempt to advance the Graph Information Bottleneck (GIB) for inductive relation prediction. Derived from the GIB principle, KG-GIB extracts critical paths which preserves task-relevant paths and blocks information from task-irrelevant paths. The extracted critical paths are expected to be more generalizable and interpretable. Extensive experiments on both synthetic and real-world datasets demonstrate the effectiveness of KG-GIB.

Memory is the core of cognition. The exploration of the memory encoding mechanism or the representation mechanism of information in the Spiking Neural Network (SNN) is the basis for the in-depth study of memory. In this paper, we study the memory encoding mechanism of multilayer SNN models from a biomimetic perspective and explore a method using the high biological likelihood of SNN to enable the network to effectively simulate memory effects. We proposed a series of heuristic neuron-growing connection algorithms and supervised network weight learning algorithms, which were applied to the unsupervised and supervised training process of the presentation layer. These methods optimized the structure of the representation layer, achieved functional differentiation of neurons, and enabled the network to generate differentiated representations for different data modes. Under our algorithm, the proposed model achieves stable convergence with identical pattern inputs, demonstrating distinct representations and sensitivities to different visual modalities. To achieve stable information expression within the network, we conducted various comparative experiments to determine diverse parameters of the complex network. This paper contributes to the development of Brain-inspired Intelligence by bridging the gap between computer science and neuroscience by using simulations to validate biological hypotheses and guide machine learning.

Medical report generation (MRG), which aims to automatically generate a textual description of a specific medical image (e.g., a chest X-ray), has recently received increasing research interest. Building on the success of image captioning, MRG has become achievable. However, generating language-specific radiology reports poses a challenge for data-driven models due to their reliance on paired image-report chest X-ray datasets, which are labor-intensive, time-consuming, and costly. In this paper, we introduce a chest X-ray benchmark dataset, namely CASIA-CXR, consisting of high-resolution chest radiographs accompanied by narrative reports originally written in French. To the best of our knowledge, this is the first public chest radiograph dataset with medical reports in this particular language. Importantly, we propose a simple yet effective multimodal encoder–decoder contextually-guided framework for medical report generation in French. We validated our framework through intra-language and cross-language contextual analysis, supplemented by expert evaluation performed by radiologists. The dataset is freely available at: https://www.casia-cxr.net/.

The rise of neural architecture search (NAS) demonstrates the deep exploration between the neural network architecture and its performance (e.g., accuracy). Many NAS methods are inefficient because they train all candidates from scratch to obtain their accuracies. Although predictor-based NAS algorithms have been vigorously developed to efficiently and accurately evaluate the performance of candidate architectures, the training of accuracy predictors still require hundreds of architectures with ground truth. To overcome this shortcoming, this paper investigates an evolutionary-based NAS method, which constructs a semi-supervised accuracy predictor to efficiently and accurately evaluate candidate architectures. A one-time extractor and strong regressors are implemented to further enhance the prediction performance of the semi-supervised accuracy predictor. Furthermore, a multi-objective approach is developed to find architectures with high ground truth in a tradeoff between high prediction accuracy and prediction confidence. Experimental results demonstrate the strong competitiveness of the proposed approach on NAS benchmarks. The code is available at https://github.com/outofstyle/SAPMNAS.

Spiking neural networks (SNNs) aim to simulate real neural networks in the human brain with biologically plausible neurons. The leaky integrate-and-fire (LIF) neuron is one of the most widely studied SNN architectures. However, it has the vanishing gradient problem when trained with backpropagation. Additionally, its neuronal parameters are often manually specified and fixed, in contrast to the heterogeneity of real neurons in the human brain. This paper proposes a gated parametric neuron (GPN) to process spatio-temporal information effectively with the gating mechanism. Compared with the LIF neuron, the GPN has two distinguishing advantages: (1) it copes well with the vanishing gradients by improving the flow of gradient propagation; and, (2) it learns spatio-temporal heterogeneous neuronal parameters automatically. Additionally, we use the same gate structure to eliminate initial neuronal parameter selection and design a hybrid recurrent neural network-SNN structure. Experiments on two spike-based audio datasets demonstrated that the GPN network outperformed several state-of-the-art SNNs, could mitigate vanishing gradients, and had spatio-temporal heterogeneous parameters. Our work shows the ability of SNNs to handle long-term dependencies and achieve high performance simultaneously.