Information Sciences最新文献

The learning and optimization of kernels in the radial basis function neural network (RBFNN) are crucial. However, in existing methods, there are issues of overfitting when learning kernel parameters. The learned kernels are also sensitive to outliers. This paper proposes a general kernel learning strategy for RBFNN called non-overlapping maximum local density support kernel learning (MLD-SKL), which contains two modules, the non-overlapping maximum local density (MLD) kernel learning module and support kernel learning (SKL) module. In the MLD kernel learning stage, the candidate set of kernels is incrementally determined based on the local density of samples. Meanwhile, it is required that the coverage ranges of kernels from different classes do not overlap with each other. This module is effective in reducing the impact of outliers. In the SKL stage, kernel importance indicator is defined to measure the importance of kernels. The learned support kernels are utilized to construct a maximum local density-driven non-overlapping radial basis function support kernel neural network (MLD-RBFSKNN). The RBFNN constructed through MLD-SKL exhibits a more compact structure. The experiments demonstrate that the proposed MLD-RBFSKNN improves accuracy in recognition task. Furthermore, while achieving superior recognition performance, the final constructed network also has the minimum number of kernels.

There have been several pioneering approaches to the extraction of attribute implications from a formal context, dating from the 1980's: the one of Guigues and Duquenne based on so-called non-redundancy nodes, another one proposed by Ganter highlighting the concept of pseudo-closed set, and the best-known one relying on the recursive computation of so-called pseudo-intents in the book by Ganter and Wille. The Guigues and Duquenne approach has never been compared in detail in the literature with the other two, although they turn out to be equivalent. This paper tries to fill this gap, proposing a unified view, hopefully more easy to grasp.

Convolutional neural networks have exhibited exceptional performance in various artificial intelligence domains, particularly in large-scale image processing tasks. However, the proliferation of network parameters and computational requirements has emerged as a significant bottleneck for the practical deployment of CNNs. In this paper, we propose a novel similarity-based filter pruning (SFP) approach for compressing convolutional neural networks, which is different from the traditional pruning method. The existing pruning methods eliminate the unimportant parameters but ignore the duplication of the reserved convolutional kernels. In the proposed SFP, kernels are clustered first according to their similarity, then the unimportant and redundant kernels are pruned in each class, which is more efficient than traditional pruning methods only based on the importance criterion. Furthermore, this paper introduces the concept of Kernel Dispersion to evaluate sparsity across distinct network layers, and proposes Distillation Fine-Tuning with Variable Temperature Coefficient to expedite convergence and enhance accuracy. The performance of the proposed similarity-based filter pruning approach is evaluated on different datasets, including CIFAR10, CIFAR100, ImageNet, and VOC. The experimental results indicate that the proposed SFP achieves approximately 1% higher accuracy at a comparable pruning rate compared to traditional state-of-the-art pruning methods.

Adversarial training is one of the most effective adversarial defense methods currently recognized. It enhances the robustness of deep neural network (DNN) classifiers by generating adversarial samples. However, current adversarial training methods cannot effectively trade off the robust accuracy and natural accuracy when training DNN classifiers, and are prone to overfit. To solve these problems, we propose Customized Adversarial Training based on Instance Loss (CATIL), aiming to improve robust accuracy and natural accuracy while alleviating overfitting. We first comprehensively consider the influencing factors of adversarial training and propose the comprehensive customization strategy (CCS), which crafts unique attack strategies for each sample, fine-tunes the classifier's decision boundary, and boosts the robustness of the DNN classifier without compromising its natural accuracy. Second, the loss adjustment strategy (LAS) is designed to update the attack strategy according to the loss value. This increases the fitting difficulty of the DNN classifier and alleviates the overfitting problem. Finally, numerous experiments show that CATIL can effectively enhance robust accuracy and alleviate the overfitting problem without significantly compromising natural accuracy. When evaluating CIFAR-10 on Wide ResNet, CATIL achieves the best performance in both natural and robust accuracy compared to all benchmarks.

The main approach to capturing user latent preferences in recommendation systems (RS) is through high-order tensor decomposition and the deep-walk method. Several key issues, if solved, could improve the performance of RS. These include enforcing the interpretation of RS in the context of sparse data completion, cold start, and interpretability, mining user latent preferences with a tensor constructed from a user-item rating matrix (RM) and a preference match mechanism based on K-nearest neighbor (KNN) similar users. In this paper, a method that integrates a hidden Markov model, meta-path, and third-order tensor (HMM-MP-TOT) is proposed. An HMM, based on the user-item RM and latent preferences from KNN users is constructed. Subsequently, the Viterbi and deep-walk methods are used to obtain a series of user-item two-dimensional MPs. Then, truncated − singular value decomposition (t-SVD) is applied to a user-item-KNN third-order tensor to obtain a better recommendation result. On average, HMM-MP-TOT obtains 94.7% precision, 80.2% recall, and 96.4% diversity.

Research on temporal knowledge graphs garners attention due to the intricate connection between facts and dynamic temporal factors. However, existing research uses timestamp as auxiliary data for representation learning and directly integrate it into facts, resulting in the inability to capture the intrinsic connections between relations under time evolution. To handle these challenges, we propose the Evolutionary Hierarchy Perception Representation (EHPR), which first leverages the Hamilton product to perform rotational transformations on relation and entity over time, aiming to learn temporal relation and temporal entity with close interactions with time information. Later, EHPR is divided into two modules: (a) Rotating the head entity towards the tail entity using temporal relation through Hamilton product to model complex patterns with quaternion rotation capabilities. (b) Adopting an evolutionary hierarchical factor to capture the differences in modulus distribution between the temporal head entity and the temporal tail entity, aiming to manage the evolutionary hierarchical information between different temporal entities. In this way, EHPR not only utilizes the rich quaternion rotation capabilities to model various relation patterns but also further enables modeling of evolutionary hierarchical patterns through evolutionary hierarchy factors. Experiments show that EHPR achieves remarkable performance on six mature benchmarks compared to state-of-the-art models. Furthermore, we successfully transferred the core idea of EHPR into complex embeddings, showcasing the framework's adaptability. Compared to complex embedding models, EHPR also demonstrates stronger expressive abilities with the Hamilton operator, surpassing the performance of complex Hermitian operator.

Link prediction involves assessing the likelihood of connections between node pairs based on various structural properties. The effectiveness of link predictors can be influenced by complex structures such as communities. Since the community structure itself has different properties that describes its characteristics, measuring the impact of these properties on the performance of link predictors presents a challenge. In this work, we aim to uncover the role of community properties and the identification of community structures on the performance of link predictors. We propose a comprehensive experimental setup to evaluate the performance of twenty-nine link predictors on real-world networks with diverse topological features, as well as on synthetic networks where we control community-dependent properties such as cohesiveness and size. We assess the performance differences between network-wide and per-community link prediction to determine whether identifying communities aids in link prediction. The results indicate that link prediction is more accurate in networks with well-defined, disjoint communities, even when these communities are not explicitly identified. Additionally, the size of the communities can influence link prediction performance if the communities are identified.

We propose an asymmetry index as a measure of degree of asymmetry of a given dataset. It provides an additional information on a dataset allowing to guide and improve any further analysis. The index reflects the intensity of the asymmetric relationships among data resulting from hierarchical data structure. Using the information retrieved by our asymmetry index, one obtains a justification and explanation of the effectiveness of the subsequent asymmetric data analysis methods, as well as helpful preparation to asymmetrizing the tools for the further analysis. The asymmetry index is based on the k-nearest neighbors graph representing the considered data. Therefore, it uses the intrinsic geometry-based information on the data, in this way, providing an insight into the data structure. Our experiments on real data are designed to verify the usefulness of the asymmetry index and the correctness of its theoretical fundamentals. In our empirical validation, we employ the symmetric and asymmetric dimensionality reduction algorithms and evaluate their results on the basis of clustering in the 2-dimensional visualization space. We test, whether our index indeed predicts the level of superiority of the asymmetric methods over their symmetric counterparts.

Online learning from the data streams is a research hotspot due to adaptive responses to real-time data arrival and fleeting. Existing approaches can only handle real-world scenarios partially due to constraints such as binary classification, complete labels, and fixed feature spaces. Learning from incomplete data streams with partial labels for multi-classification is crucial but rarely investigated due to its complexity and variability. To address this issue, we propose a novel Online Learning approach from Incomplete Data Streams with Partial Labels for Multi-classification, named OLIDS_PLM. OLIDS_PLM includes three main ideas: a) exploiting feature similarity to re-weight the most informative features in incomplete feature space (IFS) to avoid bias caused by filling in missing features, b) using self-train to label unlabeled instances and filter outliers, and c) utilizing the difference in the distribution between instances and model generated to detect concept drifts adaptively. We experimentally evaluated OLIDS_PLM and its rivals in handling the IFS, partial labels, and concept drifts to validate its effectiveness. The code is released at https://github.com/youdianlong/OLIDSPLM.

Attack detection is one of the main features required in modern defence systems. Despite the ongoing research, it remains challenging for a typical mechanism like network-based intrusion detection system (NIDS) to catch up with evolving adversarial attacks. They specifically aim to confuse a machine-learning based predictor. Without the knowledge of adversarial patterns, the best approach is generalising signatures learned from a dataset of legitimate connections and known intrusions. This work focuses on analysing non-payload traffics so that the resulting techniques can be exploited to a range of network-based applications. It investigates a novel means to deal with the problem of imbalanced classes. An optimised undersampling method is introduced to select a subset of majority-class representatives initially created through an ensemble clustering procedure. A weighted combination of criteria representing distributions within and between classes is proposed as the objective function for a global optimisation using the artificial bee colony (ABC). This approach usually outperforms its baselines and other state-of-the-art undersampling models, with ABC being more effective using the global best strategy than a random selection of solutions or an iterative greedy search. The paper also details the parameter analysis offering a heuristic guide for potential taking up of the proposed techniques.