International Journal of Machine Learning and Cybernetics最新文献

In the age of intelligent manufacturing, surface defect detection plays a pivotal role in the automated quality control of industrial products, constituting a fundamental aspect of smart factory evolution. Considering the diverse sizes and feature scales of surface defects on industrial products and the difficulty in procuring high-quality training samples, the achievement of real-time and high-quality surface defect detection through artificial intelligence technologies remains a formidable challenge. To address this, we introduce a defect detection approach grounded in the Fast Denoising Probabilistic Implicit Models. Firstly, we propose a noise predictor influenced by the spectral radius feature tensor of images. This enhancement augments the ability of generative model to capture nuanced details in non-defective areas, thus overcoming limitations in model versatility and detail portrayal. Furthermore, we present a loss function constraint based on the Perron-root. This is designed to incorporate the constraint within the representational space, ensuring the denoising model consistently produces high-quality samples. Lastly, comprehensive experiments on both the Magnetic Tile and Market-PCB datasets, benchmarked against nine most representative models, underscore the exemplary detection efficacy of our proposed approach.

Integrating convolutional neural networks (CNNs) and transformers has notably improved lightweight single image super-resolution (SISR) tasks. However, existing methods lack the capability to exploit multi-level contextual information, and transformer computations inherently add quadratic complexity. To address these issues, we propose a Joint features-Guided Linear Transformer and CNN Network (JGLTN) for efficient SISR, which is constructed by cascading modules composed of CNN layers and linear transformer layers. Specifically, in the CNN layer, our approach employs an inter-scale feature integration module (IFIM) to extract critical latent information across scales. Then, in the linear transformer layer, we design a joint feature-guided linear attention (JGLA). It jointly considers adjacent and extended regional features, dynamically assigning weights to convolutional kernels for contextual feature selection. This process garners multi-level contextual information, which is used to guide linear attention for effective information interaction. Moreover, we redesign the method of computing feature similarity within the self-attention, reducing its computational complexity to linear. Extensive experiments shows that our proposal outperforms state-of-the-art models while balancing performance and computational costs.

Multi-domain NMT aims to develop a parameter-sharing model for translating general and specific domains, such as biology, legal, etc., which often struggle with the parameter interference problem. Existing approaches typically tackle this issue by learning a domain-specific sub-network for each domain equally, but they ignore the significant data imbalance problem across domains. For instance, the training data for the general domain often outweighs the biological domain tenfold. In this paper, we observe a natural similarity between the general and specific domains, including shared vocabulary or similar sentence structure. We propose a novel parameter inheritance strategy to adaptively learn domain-specific child networks from the general domain. Our approach employs gradient similarity as the criterion for determining which parameters should be inherited or discarded between the general and specific domains. Extensive experiments on several multi-domain NMT corpora demonstrate that our method significantly outperforms several strong baselines. In addition, our method exhibits remarkable generalization performance in adapting to few-shot multi-domain NMT scenarios. Further investigations reveal that our method achieves good interpretability because the parameters learned by the child network from the general domain depend on the interconnectedness between the specific domain and the general domain.

Unsupervised feature selection (UFS), which involves selecting representative features from unlabeled high-dimensional data, has attracted much attention. Numerous self-representation-based models have been recently developed successfully for UFS. However, these models have two main problems. First, existing self-representation-based UFS models cannot effectively handle noise and outliers. Second, many graph-regularized self-representation-based UFS models typically construct a fixed graph to maintain the local structure of data. To overcome the above shortcomings, we propose a novel robust UFS model called self-representation with adaptive loss minimization via doubly stochastic graph regularization (SRALDS). Specifically, SRALDS uses an adaptive loss function to minimize the representation residual term, which may enhance the robustness of the model and diminish the effect of noise and outliers. Besides, rather than utilizing a fixed graph, SRALDS learns a high-quality doubly stochastic graph that more accurately captures the local structure of data. Finally, an efficient optimization algorithm is designed to obtain the optimal solution for SRALDS. Extensive experiments demonstrate the superior performance of SRALDS over several well-known UFS methods.

The cuckoo search algorithm (CS), an algorithm inspired by the nest-parasitic breeding behavior of cuckoos, has proved its own effectiveness as a problem-solving approach in many fields since it was proposed. Nevertheless, the cuckoo search algorithm still suffers from an imbalance between exploration and exploitation as well as a tendency to fall into local optimization. In this paper, we propose a new hybrid cuckoo search algorithm (LHCS) based on linear decreasing of populations, and in order to optimize the local search of the algorithm and make the algorithm converge quickly, we mix the solution updating strategy of the Grey Yours sincerely, wolf optimizer (GWO) and use the linear decreasing rule to adjust the calling ratio of the strategy in order to balance the global exploration and the local exploitation; Second, the addition of a specular reflection learning strategy enhances the algorithm's ability to jump out of local optima; Finally, the convergence ability of the algorithm on different intervals and the adaptive ability of population diversity are improved using a population linear decreasing strategy. The experimental results on 29 benchmark functions from the CEC2017 test set show that the LHCS algorithm has significant superiority and stability over other algorithms when the quality of all solutions is considered together. In order to further verify the performance of the proposed algorithm in this paper, we applied the algorithm to engineering problems, functional tests, and Wilcoxon test results show that the comprehensive performance of the LHCS algorithm outperforms the other 14 state-of-the-art algorithms. In several engineering optimization problems, the practicality and effectiveness of the LHCS algorithm are verified, and the design cost can be greatly reduced by applying it to real engineering problems.

Deep neural networks complete a feature extraction task by propagating the inputs through multiple modules. However, how the representations evolve with the gradient-based optimization remains unknown. Here we leverage the intrinsic dimension of the representations to study the learning dynamics and find that the training process undergoes a phase transition from expansion to compression under disparate training regimes. Surprisingly, this phenomenon is ubiquitous across a wide variety of model architectures, optimizers, and data sets. We demonstrate that the variation in the intrinsic dimension is consistent with the complexity of the learned hypothesis, which can be quantitatively assessed by the critical sample ratio that is rooted in adversarial robustness. Meanwhile, we mathematically show that this phenomenon can be analyzed in terms of the mutable correlation between neurons. Although the evoked activities obey a power-law decaying rule in biological circuits, we identify that the power-law exponent of the representations in deep neural networks predicted adversarial robustness well only at the end of the training but not during the training process. These results together suggest that deep neural networks are prone to producing robust representations by adaptively eliminating or retaining redundancies. The code is publicly available at https://github.com/cltan023/learning2022.

The majority of abstractive summarization models are designed based on the Sequence-to-Sequence(Seq2Seq) architecture. These models are able to capture syntactic and contextual information between words. However, Seq2Seq-based summarization models tend to overlook global semantic information. Moreover, there exist inconsistency between the objective function and evaluation metrics of this model. To address these limitations, a novel model named ASTCL is proposed in this paper. It integrates the neural topic model into the Seq2Seq framework innovatively, aiming to capture the text’s global semantic information and guide the summary generation. Additionally, it incorporates contrastive learning techniques to mitigate the discrepancy between the objective loss and the evaluation metrics through scoring multiple candidate summaries. On CNN/DM XSum and NYT datasets, the experimental results demonstrate that the ASTCL model outperforms the other generic models in summarization task.

Imbalanced datasets can adversely affect classifier performance. Conventional undersampling approaches may lead to the loss of essential information, while oversampling techniques could introduce noise. To address this challenge, we propose an undersampling algorithm called GLNDU (Generalized Learning Vector Quantization and Natural Nearest Neighbors-based Undersampling). GLNDU utilizes Generalized Learning Vector Quantization (GLVQ) for computing the centroids of positive and negative instances. It also utilizes the concept of Natural Nearest Neighbors to identify majority-class instances in the overlapping region of the centroids of minority-class instances. Afterwards, these majority-class instances are removed, resulting in a new balanced training dataset that is used to train a foundational classifier. We conduct extensive experiments on 29 publicly available datasets, evaluating the performance using AUC and G_mean values. GLNDU demonstrates significant advantages over established methods such as SVM, CART, and KNN across different types of classifiers. Additionally, the results of the Friedman ranking and Nemenyi post-hoc test provide additional support for the findings obtained from the experiments.