Knowledge-Based Systems最新文献_第7页

Adaptive generic prototype network with geodesic distance for cross-domain few-shot fault diagnosis 采用大地距离的自适应通用原型网络，用于跨域少量故障诊断

IF 7.2 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Knowledge-Based Systems

Pub Date : 2024-11-10 DOI: 10.1016/j.knosys.2024.112726

Yi Qin , Qijun Wen , Lv Wang , Yongfang Mao

Since the fault samples of equipment are limited and the working condition is often variable, it is valuable for researching the cross-domain few-shot fault diagnosis method to assure the safe operation of various machines. As the well-known few-shot classification approaches, the traditional prototype networks are difficult to process the complex sample distributions caused by variable operating condition data and the substantial distributional discrepancies between different machines, which seriously affects the accuracy of cross-domain fault diagnosis. To address these issues, this study proposes a new adaptive geodesic prototype network (AGPN), which can extract the category prototypes with enhanced adaptability and generalization capabilities. Firstly, a geodesic distance-driven learning strategy is developed to better measure the distance between complex samples in the embedding space. Secondly, an adaptive area prototype with a dynamic expansion coefficient is proposed, which allows for more flexible representation of different data categories. Furthermore, an adaptive momentum prototype method is put forward via a model-agnostic adaptive momentum factor, which can reduce the prototype oscillation during training and maximize the learning ability of model. The proposed AGPN is successfully applied to fault diagnosis across bearings with different operating conditions. Compared with the existing few-shot diagnosis methods, the proposed method possesses higher diagnostic accuracy and training stability, thus it is more suitable for cross-domain few-shot fault diagnosis.

由于设备的故障样本有限，且工况往往多变，因此研究跨域少量故障诊断方法对确保各种机器的安全运行具有重要价值。作为众所周知的少量故障分类方法，传统的原型网络难以处理多变工况数据带来的复杂样本分布，以及不同机器之间存在的巨大分布差异，严重影响了跨域故障诊断的准确性。针对这些问题，本研究提出了一种新的自适应大地原型网络（AGPN），它能提取出具有更强适应性和泛化能力的类别原型。首先，开发了大地距离驱动的学习策略，以更好地测量嵌入空间中复杂样本之间的距离。其次，提出了一种具有动态扩展系数的自适应区域原型，可以更灵活地表示不同的数据类别。此外，通过与模型无关的自适应动量因子，提出了一种自适应动量原型方法，可以减少训练过程中的原型振荡，最大限度地提高模型的学习能力。所提出的 AGPN 成功应用于不同运行条件下轴承的故障诊断。与现有的少量故障诊断方法相比，所提出的方法具有更高的诊断精度和训练稳定性，因此更适用于跨域少量故障诊断。

{"title":"Adaptive generic prototype network with geodesic distance for cross-domain few-shot fault diagnosis","authors":"Yi Qin , Qijun Wen , Lv Wang , Yongfang Mao","doi":"10.1016/j.knosys.2024.112726","DOIUrl":"10.1016/j.knosys.2024.112726","url":null,"abstract":"<div><div>Since the fault samples of equipment are limited and the working condition is often variable, it is valuable for researching the cross-domain few-shot fault diagnosis method to assure the safe operation of various machines. As the well-known few-shot classification approaches, the traditional prototype networks are difficult to process the complex sample distributions caused by variable operating condition data and the substantial distributional discrepancies between different machines, which seriously affects the accuracy of cross-domain fault diagnosis. To address these issues, this study proposes a new adaptive geodesic prototype network (AGPN), which can extract the category prototypes with enhanced adaptability and generalization capabilities. Firstly, a geodesic distance-driven learning strategy is developed to better measure the distance between complex samples in the embedding space. Secondly, an adaptive area prototype with a dynamic expansion coefficient is proposed, which allows for more flexible representation of different data categories. Furthermore, an adaptive momentum prototype method is put forward via a model-agnostic adaptive momentum factor, which can reduce the prototype oscillation during training and maximize the learning ability of model. The proposed AGPN is successfully applied to fault diagnosis across bearings with different operating conditions. Compared with the existing few-shot diagnosis methods, the proposed method possesses higher diagnostic accuracy and training stability, thus it is more suitable for cross-domain few-shot fault diagnosis.</div></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":"306 ","pages":"Article 112726"},"PeriodicalIF":7.2,"publicationDate":"2024-11-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142705011","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Granular intents learning via mutual information maximization for knowledge-aware recommendation 通过互信息最大化学习细粒度意图，实现知识感知推荐

IF 7.2 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Knowledge-Based Systems

Pub Date : 2024-11-09 DOI: 10.1016/j.knosys.2024.112705

Hyeongjun Yang , Yerim Lee , Gayeon Park , TaeYoung Kim , Heesun Kim , Kyong-Ho Lee , Byungkook Oh

Knowledge-aware recommender systems, which utilize knowledge graphs (KGs) to enrich item information, have been shown to improve the accuracy and explainability of recommendations. Besides, KGs are further explored to determine the intent of choosing items (i.e., the reason why users select items of interest). Conventional methods represent intents either as sets of relations in a KG or as KG entities. However, such approaches fail to fully leverage the combined information provided by both entities and relations. To address this issue, we propose a new KG-based user Intent Extraction Framework (KIEF) to capture user intents at a more fine-grained level for recommendation. Specifically, we propose a novel intent representation constructed with relation-aware entity representation, encouraging finer granularity for user intents. Furthermore, since a KG may contain noisy information that impairs the quality of user intent, it is compulsory to consider which factors in a KG are important to represent a user’s intent. Thus, we introduce global intent which are comprehensive features for the entire interactions of all users and local intent, which are empirical features of individual users from personal history. By maximizing mutual information between global and local intents, KIEF captures user preference for items. Through extensive experiments on four real-world benchmark datasets, we prove the superior performance of KIEF over the state-of-the-art and analyze interpretable explanations for understanding user intents.

知识感知推荐系统利用知识图谱（KG）来丰富项目信息，已被证明可以提高推荐的准确性和可解释性。此外，知识图谱还能进一步确定用户选择项目的意图（即用户选择感兴趣项目的原因）。传统方法将意图表示为 KG 中的关系集或 KG 实体。然而，这些方法无法充分利用实体和关系提供的综合信息。为了解决这个问题，我们提出了一种新的基于 KG 的用户意图提取框架 (KIEF)，以便在更精细的层次上捕捉用户意图，从而进行推荐。具体来说，我们提出了一种新颖的意图表示法，这种表示法采用了关系感知实体表示法，从而提高了用户意图的粒度。此外，由于幼稚园可能包含有损用户意图质量的噪声信息，因此必须考虑幼稚园中哪些因素对代表用户意图非常重要。因此，我们引入了全局意图和局部意图，前者是所有用户整个交互过程的综合特征，后者则是单个用户从个人历史中获得的经验特征。通过最大化全局意图和局部意图之间的互信息，KIEF 可以捕捉用户对物品的偏好。通过在四个真实世界基准数据集上的广泛实验，我们证明了 KIEF 优于最先进技术的性能，并分析了理解用户意图的可解释性解释。

{"title":"Granular intents learning via mutual information maximization for knowledge-aware recommendation","authors":"Hyeongjun Yang , Yerim Lee , Gayeon Park , TaeYoung Kim , Heesun Kim , Kyong-Ho Lee , Byungkook Oh","doi":"10.1016/j.knosys.2024.112705","DOIUrl":"10.1016/j.knosys.2024.112705","url":null,"abstract":"<div><div>Knowledge-aware recommender systems, which utilize knowledge graphs (KGs) to enrich item information, have been shown to improve the accuracy and explainability of recommendations. Besides, KGs are further explored to determine the intent of choosing items (i.e., the reason why users select items of interest). Conventional methods represent intents either as sets of relations in a KG or as KG entities. However, such approaches fail to fully leverage the combined information provided by both entities and relations. To address this issue, we propose a new KG-based user Intent Extraction Framework (KIEF) to capture user intents at a more fine-grained level for recommendation. Specifically, we propose a novel intent representation constructed with relation-aware entity representation, encouraging finer granularity for user intents. Furthermore, since a KG may contain noisy information that impairs the quality of user intent, it is compulsory to consider which factors in a KG are important to represent a user’s intent. Thus, we introduce global intent which are comprehensive features for the entire interactions of all users and local intent, which are empirical features of individual users from personal history. By maximizing mutual information between global and local intents, KIEF captures user preference for items. Through extensive experiments on four real-world benchmark datasets, we prove the superior performance of KIEF over the state-of-the-art and analyze interpretable explanations for understanding user intents.</div></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":"306 ","pages":"Article 112705"},"PeriodicalIF":7.2,"publicationDate":"2024-11-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142705192","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Hyperparameter recommendation via automated meta-feature selection embedded with kernel group Lasso learning 通过嵌入核群拉索学习的自动元特征选择推荐超参数

IF 7.2 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Knowledge-Based Systems

Pub Date : 2024-11-09 DOI: 10.1016/j.knosys.2024.112706

Liping Deng , MingQing Xiao

Hyperparameter recommendation via meta-learning relies on the characterization and quality of meta-features. These meta-features provide critical information about the underlying datasets but are often selected manually based on the practitioner’s experience and preference, which can be inefficient and ineffective in many applications. In this paper, we propose a novel hyperparameter recommendation approach that integrates with a Lasso-based multivariate kernel group (KGLasso) model. The developed KGLasso model automatically identifies primary meta-features through model training. By selecting the most explanatory meta-features for a specific meta-learning task, the recommendation performance becomes much more effective. Our KGLasso model builds on a group-wise generalized multivariate Lasso approach. Within this framework, we establish a minimization algorithm using a corresponding auxiliary function, which is mathematically proven to be convergent and robust. As an application, we develop a hyperparameter recommendation system using our built KGLasso model on 120 UCI datasets for the well-known support vector machine (SVM) algorithm. This system efficiently provides competent hyperparameter recommendations for new tasks. Extensive experiments, including comparisons with popular meta-learning baselines and search algorithms, demonstrate the superiority of our proposed approach. Our results highlight the benefits of integrating model learning and feature selection to construct an automated meta-learner for hyperparameter recommendation in meta-learning.

通过元学习推荐超参数依赖于元特征的特性和质量。这些元特征提供了底层数据集的关键信息，但通常是根据实践者的经验和偏好手动选择的，这在许多应用中可能是低效和无效的。在本文中，我们提出了一种新颖的超参数推荐方法，该方法与基于拉索的多元核群（KGLasso）模型相结合。所开发的 KGLasso 模型可通过模型训练自动识别主要元特征。通过为特定元学习任务选择最具解释力的元特征，推荐性能会变得更加有效。我们的 KGLasso 模型建立在分组广义多元 Lasso 方法的基础上。在此框架内，我们使用相应的辅助函数建立了最小化算法，该算法经数学证明具有收敛性和鲁棒性。在应用中，我们利用所建立的 KGLasso 模型，在 120 个 UCI 数据集上为著名的支持向量机（SVM）算法开发了一个超参数推荐系统。该系统能有效地为新任务提供胜任的超参数推荐。广泛的实验，包括与流行的元学习基线和搜索算法的比较，证明了我们提出的方法的优越性。我们的研究结果凸显了整合模型学习和特征选择来构建元学习中超参数自动推荐元学习器的优势。

{"title":"Hyperparameter recommendation via automated meta-feature selection embedded with kernel group Lasso learning","authors":"Liping Deng , MingQing Xiao","doi":"10.1016/j.knosys.2024.112706","DOIUrl":"10.1016/j.knosys.2024.112706","url":null,"abstract":"<div><div>Hyperparameter recommendation via meta-learning relies on the characterization and quality of meta-features. These meta-features provide critical information about the underlying datasets but are often selected manually based on the practitioner’s experience and preference, which can be inefficient and ineffective in many applications. In this paper, we propose a novel hyperparameter recommendation approach that integrates with a Lasso-based multivariate kernel group (KGLasso) model. The developed KGLasso model automatically identifies primary meta-features through model training. By selecting the most explanatory meta-features for a specific meta-learning task, the recommendation performance becomes much more effective. Our KGLasso model builds on a group-wise generalized multivariate Lasso approach. Within this framework, we establish a minimization algorithm using a corresponding auxiliary function, which is mathematically proven to be convergent and robust. As an application, we develop a hyperparameter recommendation system using our built KGLasso model on 120 UCI datasets for the well-known support vector machine (SVM) algorithm. This system efficiently provides competent hyperparameter recommendations for new tasks. Extensive experiments, including comparisons with popular meta-learning baselines and search algorithms, demonstrate the superiority of our proposed approach. Our results highlight the benefits of integrating model learning and feature selection to construct an automated meta-learner for hyperparameter recommendation in meta-learning.</div></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":"306 ","pages":"Article 112706"},"PeriodicalIF":7.2,"publicationDate":"2024-11-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142706025","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

TGPO-WRHNN: Two-stage Grad-CAM-guided PMRS Optimization and weighted-residual hypergraph neural network for pneumonia detection TGPO-WRHNN：两阶段 Grad-CAM 引导的 PMRS 优化和加权残差超图神经网络用于肺炎检测

IF 7.2 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Knowledge-Based Systems

Pub Date : 2024-11-09 DOI: 10.1016/j.knosys.2024.112708

Chaosheng Tang , Xinke Zhi , Junding Sun , Shuihua Wang , Yudong Zhang

Recent studies based on chest X-ray images have shown that pneumonia can be effectively detected using deep convolutional neural network methods. However, these methods tend to introduce additional noise and extract only local feature information, making it difficult to express the relationship between data objects. This study proposes a Two-stage Grad-CAM-guided pre-trained model and removal scheme (PMRS) Optimization and weighted-residual hypergraph neural network model (TGPO-WRHNN). First, our model extracts high-dimensional features using the TGPO module to capture both global and local information from an image. Second, we propose a new distance-based hypergraph construction method (DBHC) to amplify the difference between distances and better distinguish the relation between nearby and distant neighbors. Finally, we introduce a weighted-residual hypergraph convolution module (WRHC) to ensure the model maintains excellent performance, even at deeper levels. Our model was tested on a dataset of chest X-ray images of pediatric patients aged 1 to 5 years at the Guangzhou Women and Children’s Medical Centre by 10-fold cross-validation. The results showed that the method achieved a maximum accuracy of 98.97%, precision of 98.86%, recall of 98.43%, F1 score of 98.64%, and AUC of 99.78%. Compared to other existing models, our model demonstrated improvements of 0.87%, 0.86%, 0.16%, and 0.38% in terms of accuracy, precision, F1 score, and AUC, respectively.

最近基于胸部 X 光图像的研究表明，使用深度卷积神经网络方法可以有效检测肺炎。然而，这些方法往往会引入额外的噪声，并且只能提取局部特征信息，难以表达数据对象之间的关系。本研究提出了两阶段 Grad-CAM 引导的预训练模型和去除方案（PMRS）优化和加权残差超图神经网络模型（TGPO-WRHNN）。首先，我们的模型使用 TGPO 模块提取高维特征，以捕捉图像中的全局和局部信息。其次，我们提出了一种新的基于距离的超图构建方法（DBHC），以放大距离之间的差异，更好地区分近邻和远邻之间的关系。最后，我们引入了加权残差超图卷积模块（WRHC），以确保模型即使在更深的层次上也能保持出色的性能。我们的模型在广州市妇女儿童医疗中心的 1-5 岁儿科患者胸部 X 光图像数据集上进行了 10 倍交叉验证测试。结果表明，该方法的最高准确率为 98.97%，精确率为 98.86%，召回率为 98.43%，F1 分数为 98.64%，AUC 为 99.78%。与其他现有模型相比，我们的模型在准确率、精确度、F1 分数和 AUC 方面分别提高了 0.87%、0.86%、0.16% 和 0.38%。

{"title":"TGPO-WRHNN: Two-stage Grad-CAM-guided PMRS Optimization and weighted-residual hypergraph neural network for pneumonia detection","authors":"Chaosheng Tang , Xinke Zhi , Junding Sun , Shuihua Wang , Yudong Zhang","doi":"10.1016/j.knosys.2024.112708","DOIUrl":"10.1016/j.knosys.2024.112708","url":null,"abstract":"<div><div>Recent studies based on chest X-ray images have shown that pneumonia can be effectively detected using deep convolutional neural network methods. However, these methods tend to introduce additional noise and extract only local feature information, making it difficult to express the relationship between data objects. This study proposes a Two-stage Grad-CAM-guided pre-trained model and removal scheme (PMRS) Optimization and weighted-residual hypergraph neural network model (TGPO-WRHNN). First, our model extracts high-dimensional features using the TGPO module to capture both global and local information from an image. Second, we propose a new distance-based hypergraph construction method (DBHC) to amplify the difference between distances and better distinguish the relation between nearby and distant neighbors. Finally, we introduce a weighted-residual hypergraph convolution module (WRHC) to ensure the model maintains excellent performance, even at deeper levels. Our model was tested on a dataset of chest X-ray images of pediatric patients aged 1 to 5 years at the Guangzhou Women and Children’s Medical Centre by 10-fold cross-validation. The results showed that the method achieved a maximum accuracy of 98.97%, precision of 98.86%, recall of 98.43%, F1 score of 98.64%, and AUC of 99.78%. Compared to other existing models, our model demonstrated improvements of 0.87%, 0.86%, 0.16%, and 0.38% in terms of accuracy, precision, F1 score, and AUC, respectively.</div></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":"306 ","pages":"Article 112708"},"PeriodicalIF":7.2,"publicationDate":"2024-11-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142705013","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Seeking optimal and explainable deep learning models for inertial-based posture recognition 为基于惯性的姿势识别寻找最佳和可解释的深度学习模型

IF 7.2 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Knowledge-Based Systems

Pub Date : 2024-11-09 DOI: 10.1016/j.knosys.2024.112700

Diogo R. Martins , Sara M. Cerqueira , Cristina P. Santos

Deep Learning (DL) models, widely used in several domains, are often applied for posture recognition. This work researches five DL architectures for posture recognition: Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Transformer, hybrid CNN-LSTM, and hybrid CNN-Transformer. Agriculture and construction working postures were addressed as use cases, by acquiring an inertial dataset during the simulation of their typical tasks in circuits. Since model performance greatly depends on the choice of the hyperparameters, a grid search was conducted to find the optimal hyperparameters. An extensive analysis of the hyperparameter combinations’ effects is presented, identifying some general tendencies. Moreover, to unveil the black-box DL models, we applied the Gradient-weighted Class Activation Mapping (Grad-CAM) explainability method on CNN’s outputs to better understand the model’s decision-making, in terms of the most important sensors and time steps for each window output. Innovative hybrid architectures combining CNN and LSTM or Transformer encoder were implemented, by using the convolution feature maps as LSTM’s or Transformer’s inputs and fusing both subnetworks’ outputs with weights learned during the training. All architectures successfully recognized the eight posture classes, with the best model of each architecture exceeding 91.5% F1-score in the test. A top F1-score of 94.33%, with an inference time of just 0.29 ms (in a regular laptop), was achieved by a hybrid CNN-Transformer.

深度学习（DL）模型广泛应用于多个领域，经常被用于姿势识别。这项工作研究了五种用于姿势识别的深度学习架构：卷积神经网络（CNN）、长短期记忆（LSTM）、变换器、混合 CNN-LSTM 和混合 CNN-变换器。在模拟农业和建筑业典型任务的电路中，通过获取惯性数据集，将农业和建筑业的工作姿势作为用例。由于模型性能在很大程度上取决于超参数的选择，因此进行了网格搜索以找到最佳超参数。本文对超参数组合的效果进行了广泛分析，确定了一些普遍趋势。此外，为了揭示黑箱 DL 模型，我们对 CNN 的输出应用了梯度加权类激活映射（Gradient-weighted Class Activation Mapping，Grad-CAM）可解释性方法，以便从每个窗口输出最重要的传感器和时间步骤的角度，更好地理解模型的决策。通过使用卷积特征图作为 LSTM 或 Transformer 的输入，并将两个子网络的输出与训练过程中学习到的权重融合，实现了结合 CNN 和 LSTM 或 Transformer 编码器的创新混合架构。所有架构都成功识别了八个姿势类别，每个架构的最佳模型在测试中的 F1 分数都超过了 91.5%。混合 CNN-Transformer 的最高 F1 分数为 94.33%，推理时间仅为 0.29 毫秒（在普通笔记本电脑中）。

{"title":"Seeking optimal and explainable deep learning models for inertial-based posture recognition","authors":"Diogo R. Martins , Sara M. Cerqueira , Cristina P. Santos","doi":"10.1016/j.knosys.2024.112700","DOIUrl":"10.1016/j.knosys.2024.112700","url":null,"abstract":"<div><div>Deep Learning (DL) models, widely used in several domains, are often applied for posture recognition. This work researches five DL architectures for posture recognition: Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Transformer, hybrid CNN-LSTM, and hybrid CNN-Transformer. Agriculture and construction working postures were addressed as use cases, by acquiring an inertial dataset during the simulation of their typical tasks in circuits. Since model performance greatly depends on the choice of the hyperparameters, a grid search was conducted to find the optimal hyperparameters. An extensive analysis of the hyperparameter combinations’ effects is presented, identifying some general tendencies. Moreover, to unveil the black-box DL models, we applied the Gradient-weighted Class Activation Mapping (Grad-CAM) explainability method on CNN’s outputs to better understand the model’s decision-making, in terms of the most important sensors and time steps for each window output. Innovative hybrid architectures combining CNN and LSTM or Transformer encoder were implemented, by using the convolution feature maps as LSTM’s or Transformer’s inputs and fusing both subnetworks’ outputs with weights learned during the training. All architectures successfully recognized the eight posture classes, with the best model of each architecture exceeding 91.5% F1-score in the test. A top F1-score of 94.33%, with an inference time of just 0.29 ms (in a regular laptop), was achieved by a hybrid CNN-Transformer.</div></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":"306 ","pages":"Article 112700"},"PeriodicalIF":7.2,"publicationDate":"2024-11-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142705188","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Predictive modeling and anomaly detection in large-scale web portals through the CAWAL framework 通过 CAWAL 框架在大型门户网站中进行预测建模和异常检测

IF 7.2 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Knowledge-Based Systems

Pub Date : 2024-11-09 DOI: 10.1016/j.knosys.2024.112710

Özkan Canay , Ümit Kocabıçak

This study presents an approach that uses session and page view data collected through the CAWAL framework, enriched through specialized processes, for advanced predictive modeling and anomaly detection in web usage mining (WUM) applications. Traditional WUM methods often rely on web server logs, which limit data diversity and quality. Integrating application logs with web analytics, the CAWAL framework creates comprehensive session and page view datasets, providing a more detailed view of user interactions and effectively addressing these limitations. This integration enhances data diversity and quality while eliminating the preprocessing stage required in conventional WUM, leading to greater process efficiency. The enriched datasets, created by cross-integrating session and page view data, were applied to advanced machine learning models, such as Gradient Boosting and Random Forest, which are known for their effectiveness in capturing complex patterns and modeling non-linear relationships. These models achieved over 92% accuracy in predicting user behavior and significantly improved anomaly detection capabilities. The results show that this approach offers detailed insights into user behavior and system performance metrics, making it a reliable solution for improving large-scale web portals’ efficiency, reliability, and scalability.

本研究介绍了一种方法，该方法使用通过 CAWAL 框架收集的会话和页面浏览数据，并通过专门的流程加以丰富，用于网络使用挖掘（WUM）应用中的高级预测建模和异常检测。传统的网络使用挖掘方法通常依赖于网络服务器日志，这限制了数据的多样性和质量。CAWAL 框架将应用日志与网络分析相结合，创建了全面的会话和页面视图数据集，提供了更详细的用户交互视图，有效地解决了这些局限性。这种整合提高了数据的多样性和质量，同时省去了传统 WUM 所需的预处理阶段，从而提高了流程效率。通过交叉整合会话和页面视图数据创建的丰富数据集被应用于梯度提升和随机森林等先进的机器学习模型，这些模型以捕捉复杂模式和模拟非线性关系的有效性而著称。这些模型预测用户行为的准确率超过 92%，并显著提高了异常检测能力。研究结果表明，这种方法能详细洞察用户行为和系统性能指标，是提高大型门户网站效率、可靠性和可扩展性的可靠解决方案。

{"title":"Predictive modeling and anomaly detection in large-scale web portals through the CAWAL framework","authors":"Özkan Canay , Ümit Kocabıçak","doi":"10.1016/j.knosys.2024.112710","DOIUrl":"10.1016/j.knosys.2024.112710","url":null,"abstract":"<div><div>This study presents an approach that uses session and page view data collected through the CAWAL framework, enriched through specialized processes, for advanced predictive modeling and anomaly detection in web usage mining (WUM) applications. Traditional WUM methods often rely on web server logs, which limit data diversity and quality. Integrating application logs with web analytics, the CAWAL framework creates comprehensive session and page view datasets, providing a more detailed view of user interactions and effectively addressing these limitations. This integration enhances data diversity and quality while eliminating the preprocessing stage required in conventional WUM, leading to greater process efficiency. The enriched datasets, created by cross-integrating session and page view data, were applied to advanced machine learning models, such as Gradient Boosting and Random Forest, which are known for their effectiveness in capturing complex patterns and modeling non-linear relationships. These models achieved over 92% accuracy in predicting user behavior and significantly improved anomaly detection capabilities. The results show that this approach offers detailed insights into user behavior and system performance metrics, making it a reliable solution for improving large-scale web portals’ efficiency, reliability, and scalability.</div></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":"306 ","pages":"Article 112710"},"PeriodicalIF":7.2,"publicationDate":"2024-11-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142705012","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

ODDF-Net: Multi-object segmentation in 3D retinal OCTA using optical density and disease features ODDF-Net：利用光密度和疾病特征在三维视网膜 OCTA 中进行多目标分割

IF 7.2 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Knowledge-Based Systems

Pub Date : 2024-11-09 DOI: 10.1016/j.knosys.2024.112704

Chaozhi Yang , Jiayue Fan , Yun Bai , Yachuan Li , Qian Xiao , Zongmin Li , Hongyi Li , Hua Li

Automatic extraction of retinal structures, including Retinal Capillaries (RC), Retinal Arteries (RA), Retinal Veins (RV), and the Foveal Avascular Zone (FAZ), is crucial for the diagnosis and treatment of ocular diseases. This paper presents ODDF-Net, a segmentation network leveraging optical density and disease features, for the simultaneous 2D segmentation of RC, RA, RV, and FAZ in 3D Optical Coherence Tomography Angiography (OCTA). We introduce the concept of optical density to generate additional input images, enhancing the specificity for distinguishing arteries and veins. Our network employs a decoupled segmentation head to separate independent features of each object from shared features by focusing on object boundaries. Given the impact of ocular diseases on the morphology of retinal objects, we designed an auxiliary classification head and a cross-dimensional feature fusion module to model the relationship between various diseases and changes in retinal structures. Extensive experiments on two subsets of the OCTA-500 dataset demonstrate that ODDF-Net outperforms state-of-the-art methods, achieving mean intersection over union ratios of 88.17% and 82.80%. The source code is available at https://github.com/y8421036/ODDF-Net.

自动提取视网膜结构，包括视网膜毛细血管（RC）、视网膜动脉（RA）、视网膜静脉（RV）和眼窝血管区（FAZ），对于眼部疾病的诊断和治疗至关重要。本文介绍了一种利用光密度和疾病特征的分割网络 ODDF-Net，用于在三维光学相干断层扫描血管造影（OCTA）中同时对 RC、RA、RV 和 FAZ 进行二维分割。我们引入了光密度的概念来生成额外的输入图像，从而提高了区分动脉和静脉的特异性。我们的网络采用解耦分割头，通过关注对象边界，将每个对象的独立特征从共享特征中分离出来。考虑到眼部疾病对视网膜对象形态的影响，我们设计了一个辅助分类头和一个跨维特征融合模块，以模拟各种疾病与视网膜结构变化之间的关系。在 OCTA-500 数据集的两个子集上进行的大量实验表明，ODDF-Net 的表现优于最先进的方法，其平均交集比联合比分别达到了 88.17% 和 82.80%。源代码见 https://github.com/y8421036/ODDF-Net。

{"title":"ODDF-Net: Multi-object segmentation in 3D retinal OCTA using optical density and disease features","authors":"Chaozhi Yang , Jiayue Fan , Yun Bai , Yachuan Li , Qian Xiao , Zongmin Li , Hongyi Li , Hua Li","doi":"10.1016/j.knosys.2024.112704","DOIUrl":"10.1016/j.knosys.2024.112704","url":null,"abstract":"<div><div>Automatic extraction of retinal structures, including Retinal Capillaries (RC), Retinal Arteries (RA), Retinal Veins (RV), and the Foveal Avascular Zone (FAZ), is crucial for the diagnosis and treatment of ocular diseases. This paper presents ODDF-Net, a segmentation network leveraging optical density and disease features, for the simultaneous 2D segmentation of RC, RA, RV, and FAZ in 3D Optical Coherence Tomography Angiography (OCTA). We introduce the concept of optical density to generate additional input images, enhancing the specificity for distinguishing arteries and veins. Our network employs a decoupled segmentation head to separate independent features of each object from shared features by focusing on object boundaries. Given the impact of ocular diseases on the morphology of retinal objects, we designed an auxiliary classification head and a cross-dimensional feature fusion module to model the relationship between various diseases and changes in retinal structures. Extensive experiments on two subsets of the OCTA-500 dataset demonstrate that ODDF-Net outperforms state-of-the-art methods, achieving mean intersection over union ratios of 88.17% and 82.80%. The source code is available at <span><span>https://github.com/y8421036/ODDF-Net</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":"306 ","pages":"Article 112704"},"PeriodicalIF":7.2,"publicationDate":"2024-11-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142705189","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A safety realignment framework via subspace-oriented model fusion for large language models 通过面向子空间的模型融合实现大型语言模型的安全调整框架

IF 7.2 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Knowledge-Based Systems

Pub Date : 2024-11-09 DOI: 10.1016/j.knosys.2024.112701

Xin Yi , Shunfan Zheng , Linlin Wang , Xiaoling Wang , Liang He

To improve the performance of large language models (LLMs) on specific tasks, task-specific instruction fine-tuning is essential. However, this process can easily compromise the safety of a task-specific model, making it susceptible to obeying malicious instructions and generating harmful content. Current methods against fine-tuning attack usually interfere with the original fine-tuning objectives or require substantial amounts of data to realign the compromised model. To address these two major challenges, we propose reusing the initial aligned model and realigning task-specific model in the safety subspace. In this paper, we introduce a safety realignment framework through subspace-oriented model fusion (SOMF), aiming to transfer the safeguard capabilities of an initially aligned model into the current task-specific model. Our approach begins by disentangling all task vectors from the parameters of each task-specific model. We then identify safety-critical regions within these vectors by subspace masking techniques. Finally, we fuse the initial safely aligned LLM with all task vectors based on the identified safety subspace to restore the model’s safety properties. Our experiments confirm that our safety realignment framework satisfies the safety requirements of an independent task-specific model as well as traditional multitask models during their fusion. Our findings confirm that SOMF preserves safety without notably compromising performance on specific tasks while exhibiting higher data efficiency. The code is publicly available at https://github.com/xinykou/safety_realignment.

为了提高大型语言模型（LLM）在特定任务中的性能，必须对特定任务指令进行微调。然而，这一过程很容易损害特定任务模型的安全性，使其容易服从恶意指令并生成有害内容。目前针对微调攻击的方法通常会干扰原有的微调目标，或者需要大量数据来重新调整被破坏的模型。为了应对这两大挑战，我们提出了在安全子空间中重新使用初始对齐模型和重新对齐特定任务模型的建议。在本文中，我们通过面向子空间的模型融合（SOMF）引入了一个安全重新调整框架，旨在将初始对齐模型的保障能力转移到当前的特定任务模型中。我们的方法首先将所有任务向量与每个任务特定模型的参数分离开来。然后，我们通过子空间掩蔽技术识别这些向量中的安全关键区域。最后，我们根据识别出的安全子空间，将初始安全对齐的 LLM 与所有任务向量融合，以恢复模型的安全属性。我们的实验证实，我们的安全重新对齐框架既能满足独立任务特定模型的安全要求，也能满足传统多任务模型在融合过程中的安全要求。我们的研究结果证实，SOMF 既能保持安全性，又不会明显影响特定任务的性能，同时还能表现出更高的数据效率。代码可在 https://github.com/xinykou/safety_realignment 公开获取。

{"title":"A safety realignment framework via subspace-oriented model fusion for large language models","authors":"Xin Yi , Shunfan Zheng , Linlin Wang , Xiaoling Wang , Liang He","doi":"10.1016/j.knosys.2024.112701","DOIUrl":"10.1016/j.knosys.2024.112701","url":null,"abstract":"<div><div>To improve the performance of large language models (LLMs) on specific tasks, task-specific instruction fine-tuning is essential. However, this process can easily compromise the safety of a task-specific model, making it susceptible to obeying malicious instructions and generating harmful content. Current methods against fine-tuning attack usually interfere with the original fine-tuning objectives or require substantial amounts of data to realign the compromised model. To address these two major challenges, we propose reusing the initial aligned model and realigning task-specific model in the safety subspace. In this paper, we introduce a safety realignment framework through subspace-oriented model fusion (SOMF), aiming to transfer the safeguard capabilities of an initially aligned model into the current task-specific model. Our approach begins by disentangling all task vectors from the parameters of each task-specific model. We then identify safety-critical regions within these vectors by subspace masking techniques. Finally, we fuse the initial safely aligned LLM with all task vectors based on the identified safety subspace to restore the model’s safety properties. Our experiments confirm that our safety realignment framework satisfies the safety requirements of an independent task-specific model as well as traditional multitask models during their fusion. Our findings confirm that SOMF preserves safety without notably compromising performance on specific tasks while exhibiting higher data efficiency. The code is publicly available at <span><span>https://github.com/xinykou/safety_realignment</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":"306 ","pages":"Article 112701"},"PeriodicalIF":7.2,"publicationDate":"2024-11-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142705009","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

TBicomR: Event Prediction in Temporal Knowledge Graphs with Bicomplex Rotation TBicomR：时态知识图谱中的事件预测与二元旋转

IF 7.2 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Knowledge-Based Systems

Pub Date : 2024-11-08 DOI: 10.1016/j.knosys.2024.112711

Ngoc-Trung Nguyen , Chi Tran , Thanh Le

Temporal knowledge graphs (TKGs) capture relationships and entities evolving over time, making event prediction a challenging task due to the complex temporal and relational dynamics. In this work, we propose BiCoTime, a novel model using bicomplex embeddings to represent entities, relations, and time. While quaternions capture asymmetric relations through non-commutativity, bicomplex numbers provide a commutative algebraic structure, ideal for modeling both symmetric and asymmetric relations. Unlike quaternions, bicomplex embeddings maintain interpretability in symmetric relations while preserving key algebraic properties like distributivity. Temporal rotations further enhance BiCoTime's ability to model the interaction between relations and time, capturing how entities and relationships evolve. This combination of bicomplex embeddings and temporal rotations ensures a more interpretable and accurate modeling of TKGs. Our experiments show that TBiComR achieved a 21% improvement in Mean Reciprocal Rank (MRR) on the ICEWS14 dataset, which emphasizes time points, and a 15% improvement on the YAGO11k dataset, which focuses on time spans. The choice of bicomplex numbers balances computational complexity and expressive power, offering efficient training and better predictive performance compared to models using quaternions or octonions.

时态知识图谱（TKGs）捕捉了随时间演变的关系和实体，由于复杂的时间和关系动态，事件预测成为一项具有挑战性的任务。在这项工作中，我们提出了 BiCoTime，这是一种使用二元嵌入来表示实体、关系和时间的新型模型。四元数通过非交换性捕捉非对称关系，而二元数提供了交换代数结构，是对称和非对称关系建模的理想选择。与四元数不同，二复数嵌入保持了对称关系的可解释性，同时保留了关键的代数特性，如分布性。时间旋转进一步增强了 BiCoTime 对关系和时间之间的交互作用进行建模的能力，从而捕捉到实体和关系是如何演变的。这种双复嵌入和时间旋转的结合确保了对 TKGs 建模的可解释性和准确性。我们的实验表明，在强调时间点的 ICEWS14 数据集上，TBiComR 的平均互斥等级 (MRR) 提高了 21%，而在强调时间跨度的 YAGO11k 数据集上，TBiComR 的平均互斥等级 (MRR) 提高了 15%。与使用四元数或八元数的模型相比，二元数的选择平衡了计算复杂性和表现力，提供了高效的训练和更好的预测性能。

{"title":"TBicomR: Event Prediction in Temporal Knowledge Graphs with Bicomplex Rotation","authors":"Ngoc-Trung Nguyen , Chi Tran , Thanh Le","doi":"10.1016/j.knosys.2024.112711","DOIUrl":"10.1016/j.knosys.2024.112711","url":null,"abstract":"<div><div>Temporal knowledge graphs (TKGs) capture relationships and entities evolving over time, making event prediction a challenging task due to the complex temporal and relational dynamics. In this work, we propose BiCoTime, a novel model using bicomplex embeddings to represent entities, relations, and time. While quaternions capture asymmetric relations through non-commutativity, bicomplex numbers provide a commutative algebraic structure, ideal for modeling both symmetric and asymmetric relations. Unlike quaternions, bicomplex embeddings maintain interpretability in symmetric relations while preserving key algebraic properties like distributivity. Temporal rotations further enhance BiCoTime's ability to model the interaction between relations and time, capturing how entities and relationships evolve. This combination of bicomplex embeddings and temporal rotations ensures a more interpretable and accurate modeling of TKGs. Our experiments show that TBiComR achieved a 21% improvement in Mean Reciprocal Rank (MRR) on the ICEWS14 dataset, which emphasizes time points, and a 15% improvement on the YAGO11k dataset, which focuses on time spans. The choice of bicomplex numbers balances computational complexity and expressive power, offering efficient training and better predictive performance compared to models using quaternions or octonions.</div></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":"306 ","pages":"Article 112711"},"PeriodicalIF":7.2,"publicationDate":"2024-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142705186","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A lightweight Future Skeleton Generation Network(FSGN) based on spatio-temporal encoding and decoding 基于时空编码和解码的轻量级未来骨架生成网络（FSGN）

IF 7.2 1区计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Knowledge-Based Systems

Pub Date : 2024-11-08 DOI: 10.1016/j.knosys.2024.112717

Tingyu Liu , Chenyi Weng , Jun Huang , Zhonghua Ni

Since early warning in industrial applications is far more valuable than post-event analysis, human activity prediction based on partially observed skeleton sequences has become a popular research area. Recent studies focus on building complex deep learning networks to generate accurate future skeleton data, but overlook the requirement for timeliness. Different from such frame-by-frame generation methods, we propose a Future Skeleton Generation Network (FSGN) based on spatio-temporal encoding and decoding framework. Firstly, we design a dynamically regulated input module to ensure equal-length input of partially observed data, and set modules like discrete cosine transform(DCT) and low-pass filtering(LPF) to filter important information. Then, we employ an improved multi-layer perceptron(MLP) structure as the basic computational unit for the encoding and decoding framework to extract spatio-temporal information, and propose using multi-dimensional motion error of human skeleton to form the loss function. Finally, we use an output module symmetrical to the input module to achieve the generation of future activity data. Results show that the proposed FSGN achieves fewer parameters(0.12 M) and higher generation accuracy, which can effectively provide future information for human activity prediction tasks.

由于工业应用中的早期预警远比事后分析更有价值，因此基于部分观察到的骨骼序列进行人类活动预测已成为一个热门研究领域。最近的研究侧重于构建复杂的深度学习网络来生成准确的未来骨架数据，但忽略了对时效性的要求。与这种逐帧生成方法不同，我们提出了一种基于时空编码和解码框架的未来骨架生成网络（FSGN）。首先，我们设计了一个动态调节输入模块，以确保部分观测数据的等长输入，并设置离散余弦变换（DCT）和低通滤波（LPF）等模块来过滤重要信息。然后，我们采用改进的多层感知器（MLP）结构作为编码和解码框架的基本计算单元来提取时空信息，并提出利用人体骨骼的多维运动误差来形成损失函数。最后，我们使用与输入模块对称的输出模块来实现未来活动数据的生成。结果表明，所提出的 FSGN 实现了更少的参数（0.12 M）和更高的生成精度，可以有效地为人类活动预测任务提供未来信息。

{"title":"A lightweight Future Skeleton Generation Network(FSGN) based on spatio-temporal encoding and decoding","authors":"Tingyu Liu , Chenyi Weng , Jun Huang , Zhonghua Ni","doi":"10.1016/j.knosys.2024.112717","DOIUrl":"10.1016/j.knosys.2024.112717","url":null,"abstract":"<div><div>Since early warning in industrial applications is far more valuable than post-event analysis, human activity prediction based on partially observed skeleton sequences has become a popular research area. Recent studies focus on building complex deep learning networks to generate accurate future skeleton data, but overlook the requirement for timeliness. Different from such frame-by-frame generation methods, we propose a Future Skeleton Generation Network (FSGN) based on spatio-temporal encoding and decoding framework. Firstly, we design a dynamically regulated input module to ensure equal-length input of partially observed data, and set modules like discrete cosine transform(DCT) and low-pass filtering(LPF) to filter important information. Then, we employ an improved multi-layer perceptron(MLP) structure as the basic computational unit for the encoding and decoding framework to extract spatio-temporal information, and propose using multi-dimensional motion error of human skeleton to form the loss function. Finally, we use an output module symmetrical to the input module to achieve the generation of future activity data. Results show that the proposed FSGN achieves fewer parameters(0.12 M) and higher generation accuracy, which can effectively provide future information for human activity prediction tasks.</div></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":"306 ","pages":"Article 112717"},"PeriodicalIF":7.2,"publicationDate":"2024-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142705010","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0