Neural Processing Letters最新文献

A massive amount of information explosion over the internet has caused a possible difficulty of information overload. To overcome this, Recommender systems are systematic tools that are rapidly being employed in several domains such as movies, travel, E-commerce, and music. In the existing research, several methods have been proposed for single-user modeling, however, the massive rise of social connections potentially increases the significance of group recommender systems (GRS). A GRS is one that jointly recommends a list of items to a collection of individuals based on their interests. Moreover, the single-user model poses several challenges to recommender systems such as data sparsity, cold start, and long tail problems. On the contrary hand, another hotspot for group-based recommendation is the modeling of user preferences and interests based on the groups to which they belong using effective aggregation strategies. To address such issues, a novel “KGR” group recommender system based on user-trust relations is proposed in this study using kernel mapping techniques. In the proposed model, user-trust networks or relations are exploited to generate trust-based groups of users which is one of the important behavioral and social aspects. More precisely, in KGR the group kernels and group residual matrices are exploited as well as seeking a multi-linear mapping between encoded vectors of group-item interactions and probability density function indicating how groups will rate the items. Moreover, to emphasize the relevance of individual preferences of users in a group to which they belong, a hybrid approach is also suggested in which group kernels and individual user kernels are merged as additive and multiplicative models. Furthermore, the proposed KGR is validated on two different trust-based datasets including Film Trust and CiaoDVD. In addition, KGR outperforms with an RMSE value of 0.3306 and 0.3013 on FilmTrust and CiaoDVD datasets which are lower than the 1.8176 and 1.1092 observed with the original KMR.

We propose a simple image high-perception super-resolution reconstruction method based on multi-layer feature fusion model with adaptive compression and parameter tuning. The aim is to further balance the high and low-frequency information of an image, enrich the detailed texture to improve perceptual quality, and improve the adaptive optimization and generalization of the model in the process of super-resolution reconstruction. First, an effective multi-layer fusion super-resolution (MFSR) basic model is constructed by the design of edge enhancement, refine layering, enhanced super-resolution generative adversarial network and other sub-models, and effective multi-layer fusion. This further enriches the image representation of features of different scales and depths and improves the feature representation of high and low-frequency information in a balanced way. Next, a total loss function of the generator is constructed with adaptive parameter tuning performance. The overall adaptability of the model is improved through adaptive weight distribution and fusion of content loss, perceptual loss, and adversarial loss, and improving the error while reducing the edge enhancement model. Finally, a fitness function with the evaluation perceptual function as the optimization strategy is constructed, and the model compression and adaptive tuning of MFSR are carried out based on the multi-mechanism fusion strategy. Consequently, the construction of the adaptive MFSR model is realized. Adaptive MFSR can maintain high peak signal to noise ratio and structural similarity on the test sets Set5, Set14, and BSD100, and achieve high-quality reconstructed images with low learned perceptual image patch similarity and perceptual index, while having good generalization capabilities.

Artificial neural network (NN) architecture design is a nontrivial and time-consuming task that often requires a high level of human expertise. Neural architecture search (NAS) serves to automate the design of NN architectures and has proven to be successful in automatically finding NN architectures that outperform those manually designed by human experts. NN architecture performance can be quantified based on multiple objectives, which include model accuracy and some NN architecture complexity objectives, among others. The majority of modern NAS methods that consider multiple objectives for NN architecture performance evaluation are concerned with automated feed forward NN architecture design, which leaves multi-objective automated recurrent neural network (RNN) architecture design unexplored. RNNs are important for modeling sequential datasets, and prominent within the natural language processing domain. It is often the case in real world implementations of machine learning and NNs that a reasonable trade-off is accepted for marginally reduced model accuracy in favour of lower computational resources demanded by the model. This paper proposes a multi-objective evolutionary algorithm-based RNN architecture search method. The proposed method relies on approximate network morphisms for RNN architecture complexity optimisation during evolution. The results show that the proposed method is capable of finding novel RNN architectures with comparable performance to state-of-the-art manually designed RNN architectures, but with reduced computational demand.

Multiple hidden layers in deep neural networks perform non-linear transformations, enabling the extraction of meaningful features and the identification of relationships between input and output data. However, the gap between the training and real-world data can result in network overfitting, prompting the exploration of various preventive methods. The regularization technique called ’dropout’ is widely used for deep learning models to improve the training of robust and generalized features. During the training phase with dropout, neurons in a particular layer are randomly selected to be ignored for each input. This random exclusion of neurons encourages the network to depend on different subsets of neurons at different times, fostering robustness and reducing sensitivity to specific neurons. This study introduces a novel approach called random focusing, departing from complete neuron exclusion in dropout. The proposed random focusing selectively highlights random neurons during training, aiming for a smoother transition between training and inference phases while keeping network architecture consistent. This study also incorporates Jensen–Shannon Divergence to enhance the stability and efficacy of the random focusing method. Experimental validation across tasks like image classification and semantic segmentation demonstrates the adaptability of the proposed methods across different network architectures, including convolutional neural networks and transformers.

The vision transformer(ViT), pre-trained on large datasets, outperforms convolutional neural networks (CNN) in computer vision(CV). However, if not pre-trained, the transformer architecture doesn’t work well on small datasets and is surpassed by CNN. Through analysis, we found that:(1) the division and processing of tokens in the ViT discard the marginalized information between token. (2) the isolated multi-head self-attention (MSA) lacks prior knowledge. (3) the local inductive bias capability of stacked transformer block is much inferior to that of CNN. We propose a novel architecture for small data paradigms without pre-training, named Add-Vit, which uses progressive tokenization with feature supplementation in patch embedding. The model’s representational ability is enhanced by using a convolutional prediction module shortcut to connect MSA and capture local features as additional representations of the token. Without the need for pre-training on large datasets, our best model achieved 81.25(%) accuracy when trained from scratch on the CIFAR-100.

Deep neural networks perform better than shallow neural networks, but the former tends to be deeper or wider, introducing large numbers of parameters and computations. We know that networks that are too wide have a high risk of overfitting and networks that are too deep require a large amount of computation. This paper proposed a narrow-deep ResNet, increasing the depth of the network while avoiding other issues caused by making the network too wide, and used the strategy of knowledge distillation, where we set up a trained teacher model to train an unmodified, wide, and narrow-deep ResNet that allows students to learn the teacher’s output. To validate the effectiveness of this method, it is tested on Cifar-100 and Pascal VOC datasets. The method proposed in this paper allows a small model to have about the same accuracy rate as a large model, while dramatically shrinking the response time and computational effort.

Nested named entity recognition (NNER) aims to identify potentially overlapping named entities. Sequence labeling method and span-based method are two commonly used methods in nested named entity recognition. However, the linear structure of sequence labeling method results in relatively poor performance, and span-based method requires traversing all spans, which brings very high time complexity. All of them fail to effectively leverage the positional dependencies between internal and external entities. In order to improve these issues, this paper proposed a nested entity recognition method based on Multidimensional Features and Fuzzy Localization (MFFL). Firstly, this method adopted the shared encoding that fused three features of characters, words, and parts of speech to obtain a multidimensional feature vector representation of the text and obtained rich semantic information in the text. Secondly, we proposed to use the fuzzy localization to assist the model in pinpointing the potential locations of entities. Finally, in the entity classification, it used a window to expand the sub-sequence and enumerate possible candidate entities and predicted the classification labels of these candidate entities. In order to alleviate the problem of error propagation and effectively learn the correlation between fuzzy localization and classification labels, we adopted multi-task learning strategy. This paper conducted several experiments on two public datasets. The experimental results showed that the proposed method achieves ideal results in both nested entity recognition and non-nested entity recognition tasks, and significantly reduced the time complexity of nested entity recognition.

The era of big data provides a platform for high-precision RUL prediction, but the existing RUL prediction methods, which effectively extract key degradation information, remain a challenge. Existing methods ignore the influence of sensor and degradation moment variability, and instead assign weights to them equally, which affects the final prediction accuracy. In addition, convolutional networks lose key information due to downsampling operations and also suffer from the drawback of insufficient feature extraction capability. To address these issues, the two-layer attention mechanism and the Inception module are embedded in the capsule structure (mai-capsule model) for lifetime prediction. The first layer of the channel attention mechanism (CAM) evaluates the influence of various sensor information on the forecast; the second layer adds a time-step attention (TSAM) mechanism to the LSTM network to weigh the contribution of different moments of the engine's whole life cycle to the prediction, while weakening the influence of environmental noise on the prediction. The Inception module is introduced to perform multi-scale feature extraction on the weighted data to capture the degradation information to the maximum extent. Lastly, we are inspired to employ the capsule network to capture important position information of high and low-dimensional features, given its capacity to facilitate a more effective rendition of the overall features of the time-series data. The efficacy of the suggested model is assessed against other approaches and verified using the publicly accessible C-MPASS dataset. The end findings demonstrate the excellent prediction precision of the suggested approach.

Recommender systems have become indispensable for addressing information overload for micro-video services. They are used to characterize users’ preferences from their historical interactions and recommend micro-videos accordingly. Existing works largely leverage the multi-modal contents of micro-videos to enhance recommendation performance. However, limited efforts have been made to understand users’ complex behavior patterns, including their long- and short-term interests, as well as their temporal diversity preferences. In micro-video recommendation scenarios, users tend to have both stable long-term interests and dynamic short-term interests, and may feel tired after incessantly receiving numerous similar recommendations. In this paper, we propose a Temporal Diversity-aware micro-video recommender (TD-VideoRec) for user behavior modeling, simultaneously capturing users’ long- and short-term preferences. Specifically, we first adopt a user-centric attention mechanism to cope with long-term interests. Then, we utilize an attention network on top of a long-short term memory network to obtain users’ short-term interests. Finally, a temporal diversity coefficient is introduced to characterize the temporal diversity preferences of users’ click behaviors. The value of the coefficient depends on the distinction between users’ long- and short-term interests extracted by vector orthogonal projection. Extensive experiments on two real-world datasets demonstrate that TD-VideoRec outperforms state-of-the-art methods.

Lower versions of EfficientDet (such as D0, D1) have smaller network structures and parameter sizes, but lower detection accuracy. Higher versions exhibit higher accuracy, but the increase in network complexity poses challenges for real-time processing and hardware requirements. To meet the higher accuracy requirements under limited computational resources, this paper introduces SpanEffiDet based on the channel adaptive frequency filter (CAFF) and the Span-Path Bidirectional Feature Pyramid structure. Firstly, the CAFF module proposed in this paper realizes the frequency domain transformation of channel information through Fourier transform and effectively extracts the key features through semantic adaptive frequency filtering, thus, eliminating channel redundant information of EfficientNet. Simultaneously, the module has the ability to compute the weights across the channels and at fine granularity, and capture the detailed information of element features. Secondly, a two-way characteristic pyramid network multi-level cross-BIFPN, which can achieve multi-layer and multi-nodes, is proposed to build cross-level information transmission to incorporate both semantic and positional information of the target. This design enables the network to more effectively detect objects with significant size differences in complex environments. Finally, by introducing generalized focal Loss V2, reliable localization quality estimation scores are predicted from the distribution statistics of bounding boxes, thereby improving localization accuracy. The experimental results indicate that on the MS COCO dataset, SpanEffiDet-D0 achieved an AP improvement of 3.3% compared to the original EfficientDet series algorithms. Similarly, on the PASCAL VOC2007 and 2012 datasets, the mAP of SpanEffiDet-D0 is respectively 1.66 and 2.65% higher than that of EfficientDet-D0.