Knowledge-Based Systems最新文献

The Vision Transformer (ViT) outperforms Convolutional Neural Networks (CNNs) but at the cost of significantly higher computational demands. Knowledge Distillation (KD) has shown promise in compressing complex networks by transferring knowledge from a large pre-trained model to a smaller one. However, current KD methods for ViT often rely on CNNs as teachers or neglect the importance of class token ([CLS]) information, resulting in ineffective distillation of ViT’s unique knowledge. In this paper, we propose Adaptive Class token Knowledge Distillation ([CLS]-KD), which fully exploits information from the class token and patches in ViT. For class embedding (CLS) distillation, the intermediate CLS of the student model is aligned with the corresponding CLS of the teacher model through a projector. Furthermore, we introduce CLS-patch attention map distillation, where an attention map between the CLS and patch embeddings is generated and matched at each layer. This empowers the student model to learn how to dynamically extract patch embedding information into the CLS under teacher guidance. Finally, we propose Adaptive Layer-wise Distillation (ALD) to mitigate the imbalance in distillation effects varying with the depth of layers. This method assigns greater weight to the losses in layers where the training discrepancies between the teacher and student models are larger during distillation. Through these strategies, [CLS]-KD consistently surpasses existing state-of-the-art methods on the ImageNet-1K dataset across various teacher–student configurations. Furthermore, the proposed method demonstrates its generalization capability through transfer learning experiments on the CIFAR-10, CIFAR-100, and CALTECH-256 datasets.

Decentralized resource-constrained multi-project scheduling (DRCMPSP) is becoming increasingly common in construction, supply chains, and many other industrial disciplines. DRCMPSP faces difficult decisions in resolving resource conflicts to generate a baseline schedule to optimize global objectives. We propose an agent-based approach to address the DRCMPSP based on two global objectives: average project delay and total project delay. A heuristic based on the priority rule (PR) is developed to coordinate the global resource allocation. A comprehensive analysis of 30 PRs was conducted on 16,000 portfolios containing 48,000 projects . We confirmed that using the same PR to allocate global resources on all occasions often results in unnecessarily poor performance. The best PR depends on project and portfolio characteristics such as serial/parallel indicators, global resource distribution, and tightness. Moreover, the best PR differs from various perspectives (e.g., projects and portfolios). We summarized our results in three decision tables and further distilled these results for practical use, which only provide a rough estimate of the project and portfolio characteristics.

Multi-Object Tracking (MOT) holds significant potential for enhancing the analysis of sporting events. Traditional MOT models are primarily designed for pedestrian-centric scenarios with static cameras and linear motion patterns. However, the dynamic environment of sports presents unique challenges: (i) significant camera movements and dynamic focal length adjustments cause abrupt changes in player positions across frames; (ii) player trajectories are nonlinear and influenced by game dynamics, resulting in complex, rapid movements complicated by erratic camera motion; and (iii) issues like image blurring, occlusion, and similar player appearances challenge visual identification robustness. These factors create substantial obstacles for standard tracking algorithms. To address these challenges, we introduce DCTracker, a specialized MOT system for robust performance in soccer matches. Our approach enhances the conventional Kalman filter by integrating a bird’s-eye view via homography and inter-frame registration for the broadcast view, termed the dual-view Kalman filter (DVKF). This method leverages context from both perspectives to enrich the estimation model with multi-state vectors for each object, mitigating the impact of camera motion and nonlinear trajectories. We also introduce the cascade selection module (CSM), which optimizes the strengths of each perspective by dynamically adjusting their influence using spatial topological relationships among players. The CSM creates an adaptive cost matrix that effectively manages visual issues from blurring and occlusion. The efficacy of our method is demonstrated through state-of-the-art performance on the SoccerNet-Tracking test set and the SportsMOT-soccer validation split, highlighting its robustness across diverse venues and challenging player trajectories.

Artificial Intelligence (AI) is seen as a disruptive solution to the ever increasing security threats on network infrastructures. To automate the process of defending networked environments from such threats, approaches such as Reinforcement Learning (RL) have been used to train agents in cyber adversarial games. One primary challenge is how contextual information could be integrated into RL models to create agents which adapt their behaviour to adversarial posture. Two desirable characteristics identified for such models are that they should be interpretable and causal.

To address this challenge, we propose an approach through the integration of a causal rewards model with a modified Proximal Policy Optimisation (PPO) agent in Meta’s MBRL-Lib framework. Our RL agents are trained and evaluated against a range of cyber-relevant scenarios in the Dstl YAWNING-TITAN (YT) environment. We have constructed and experimented with two types of reward functions to facilitate the agent’s learning process. Evaluation metrics include, among others, games won by the defence agent (blue wins), episode length, healthy nodes and isolated nodes.

Results show that, over all scenarios, our causally aware agent achieves better performance than causally-blind state-of-the-art benchmarks in these scenarios for the above evaluation metrics. In particular, with our proposed High Value Target (HVT) rewards function, which aims not to disrupt HVT nodes, the number of isolated nodes is improved by 17% and 18% against the model-free and Neural Network (NN) model-based agents across all scenarios. More importantly, the overall performance improvement for the blue wins metric exceeded that of model-free and NN model-based agents by 40% and 17%, respectively, across all scenarios.

One of the main challenges in applying preference-based approaches to many-objective optimization problems is that decision makers (DMs) initially have only a vague notion of the solution they want and can obtain. In this paper, we propose an interactive approach that aids DMs in articulating a preference model in a progressive way. The quality of a solution is determined in terms of its “preference closeness” to an aspiration point, which is a subjective concept that can be outlined by the DM. Our proposal is based on compensatory fuzzy logic, which allows for the construction of predicates that are expressed in language that is close to natural. One main advantage is that the model can be optimized via metaheuristics, and we utilize an ant colony optimization algorithm for this. Our model complies with the principles of hybrid augmented intelligence, not only because the algorithm is enriched with knowledge from the DM, but also because the DM also learns the concept of “preference closeness” throughout the process. The proposed model is validated on benchmarks with five and 10 objective functions, and is compared with two state-of-the-art algorithms. Our approach allows for better convergence to the best compromise solutions. The advantages of our approach are supported by statistical tests of the results.

Sequential recommendation (SR) has become a research hotspot recently. In our research, we observe that most existing SR models only leverage each user’s own interaction sequence to make recommendation. We argue that leveraging global contextual information across different interaction sequences could enrich item representations and thereby improve recommendation performance. To achieve this, we formulate a global graph from different sequences, providing global contextual information for each sequence. Specifically, we propose to conduct graph contrastive learning on a subgraph sampled from the global graph and a local sequence graph built from each sequence to augment item representations within each sequence. At the same time, we observe that structural dependencies, referring to relationships between items based on the graphic structure, can be extracted from the constructed global graph. Capturing structural dependencies between items may enrich the item representations. To leverage structural dependencies, we propose a new attention mechanism referred to as the Jaccard attention. While prevalent Transformer-based SR models capture semantic dependencies, referring to relationships between items based on item embeddings, in a sequence through self-attention. Therefore, it is beneficial to capture both semantic and structural dependencies between items in a sequence to further enrich item representations. Specifically, we employ two sequence encoders based on the self-attention and the proposed Jaccard attention to capture semantic and structural dependencies between items in a sequence, respectively. Overall, we propose a Global Contextual enhanced Structural-aware Transformer (GC-ST) for SR. Extensive experiments carried out on three widely used datasets demonstrate the effectiveness of GC-ST.

Person re-identification (Re-ID) has achieved tremendous success in the fields of computer vision and security. However, Re-ID models are susceptible to adversarial examples, which are crafted by introducing imperceptible perturbations to benign person images. These adversarial examples often display high success rates in white-box settings but their transferability to black-box settings is relatively low. To improve the transferability of adversarial examples, this paper proposes a novel approach called the adaptive gradient similarity attack (AGS), which encompasses two essential components: gradient similarity and enhanced second moment. Specifically, a gradient similarity modulation is established to better harness the information in the neighborhood of the adjacent input, which can adaptively correct the update direction. Additionally, this paper formulates an enhanced second moment to adjust the update step at each iteration to address the issue of poor transferability. Extensive experiments confirm that AGS achieves the best performance compared with the state-of-the-art gradient-based attacks. Moreover, AGS is a versatile approach that can be integrated with existing input transformation attack techniques. Code is available at https://github.com/ZezeTao/similar_Attack4.

Defect recognition of flat metals is paramount for ensuring quality control during the production process. However, the diverse origins of metal surface damage, ranging from mechanical impacts to chemical corrosion, and the resulting varied morphology and scale of surface defects, particularly numerous microdefects and elongated defects with high aspect ratios, complicate defect recognition. Existing methods fail to select the most beneficial features during extraction and commonly lose critical feature information during gradient sampling. To overcome these challenges, we propose a lightweight network to optimize feature screening for defect recognition. First, we propose a deformable context–guided block that employs deformable convolution to dynamically adapt the perception of the spatial context, providing precise guidance of relevant semantic information in complex surface textures. Second, we develop a content-aware feature compression block that implements adaptive weighting of features, which significantly reduces information loss during the downsampling stage. Finally, we introduce an intra-scale feature interaction transformer block, which optimizes high-order semantic features to enhance the accuracy and reliability of defect detection. Experimental validation on the NEU-DET, APS-DET, and GC10-DET datasets demonstrated significant improvements in the detection accuracy and parameter efficiency, confirming the proposed method's robust generalizability.