Pub Date : 2024-11-07DOI: 10.1109/TBDATA.2024.3489427
Faqian Guan;Tianqing Zhu;Hui Sun;Wanlei Zhou;Philip S. Yu
Graph data contains rich node features and unique edge information, which have been applied across various domains, such as citation networks or recommendation systems. Graph Neural Networks (GNNs) are specialized for handling such data and have shown impressive performance in many applications. However, GNNs may contain of sensitive information and susceptible to privacy attacks. For example, link stealing is a type of attack in which attackers infer whether two nodes are linked or not. Previous link stealing attacks primarily relied on posterior probabilities from the target GNN model, neglecting the significance of node features. Additionally, variations in node classes across different datasets lead to different dimensions of posterior probabilities. The handling of these varying data dimensions posed a challenge in using a single model to effectively conduct link stealing attacks on different datasets. To address these challenges, we introduce Large Language Models (LLMs) to perform link stealing attacks on GNNs. LLMs can effectively integrate textual features and exhibit strong generalizability, enabling attacks to handle diverse data dimensions across various datasets. We design two distinct LLM prompts to effectively combine textual features and posterior probabilities of graph nodes. Through these designed prompts, we fine-tune the LLM to adapt to the link stealing attack task. Furthermore, we fine-tune the LLM using multiple datasets and enable the LLM to learn features from different datasets simultaneously. Experimental results show that our approach significantly enhances the performance of existing link stealing attack tasks in both white-box and black-box scenarios. Our method can execute link stealing attacks across different datasets using only a single model, making link stealing attacks more applicable to real-world scenarios.
{"title":"Large Language Models for Link Stealing Attacks Against Graph Neural Networks","authors":"Faqian Guan;Tianqing Zhu;Hui Sun;Wanlei Zhou;Philip S. Yu","doi":"10.1109/TBDATA.2024.3489427","DOIUrl":"https://doi.org/10.1109/TBDATA.2024.3489427","url":null,"abstract":"Graph data contains rich node features and unique edge information, which have been applied across various domains, such as citation networks or recommendation systems. Graph Neural Networks (GNNs) are specialized for handling such data and have shown impressive performance in many applications. However, GNNs may contain of sensitive information and susceptible to privacy attacks. For example, link stealing is a type of attack in which attackers infer whether two nodes are linked or not. Previous link stealing attacks primarily relied on posterior probabilities from the target GNN model, neglecting the significance of node features. Additionally, variations in node classes across different datasets lead to different dimensions of posterior probabilities. The handling of these varying data dimensions posed a challenge in using a single model to effectively conduct link stealing attacks on different datasets. To address these challenges, we introduce Large Language Models (LLMs) to perform link stealing attacks on GNNs. LLMs can effectively integrate textual features and exhibit strong generalizability, enabling attacks to handle diverse data dimensions across various datasets. We design two distinct LLM prompts to effectively combine textual features and posterior probabilities of graph nodes. Through these designed prompts, we fine-tune the LLM to adapt to the link stealing attack task. Furthermore, we fine-tune the LLM using multiple datasets and enable the LLM to learn features from different datasets simultaneously. Experimental results show that our approach significantly enhances the performance of existing link stealing attack tasks in both white-box and black-box scenarios. Our method can execute link stealing attacks across different datasets using only a single model, making link stealing attacks more applicable to real-world scenarios.","PeriodicalId":13106,"journal":{"name":"IEEE Transactions on Big Data","volume":"11 4","pages":"1879-1893"},"PeriodicalIF":7.5,"publicationDate":"2024-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144597809","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-06DOI: 10.1109/TBDATA.2024.3489412
Yuxin Guo;Deyu Bo;Cheng Yang;Zhiyuan Lu;Zhongjian Zhang;Jixi Liu;Yufei Peng;Chuan Shi
The history of artificial intelligence (AI) has witnessed the significant impact of high-quality data on various deep learning models, such as ImageNet for AlexNet and ResNet. Recently, instead of designing more complex neural architectures as model-centric approaches, the attention of AI community has shifted to data-centric ones, which focuses on better processing data to strengthen the ability of neural models. Graph learning, which operates on ubiquitous topological data, also plays an important role in the era of deep learning. In this survey, we comprehensively review graph learning approaches from the data-centric perspective, and aim to answer three crucial questions: (1) when to modify graph data, (2) what part of the graph data needs modification to unlock the potential of various graph models, and (3) how to safeguard graph models from problematic data influence. Accordingly, we propose a novel taxonomy based on the stages in the graph learning pipeline, and highlight the processing methods for different data structures in the graph data, i.e., topology, feature and label. Furthermore, we analyze some potential problems embedded in graph data and discuss how to solve them in a data-centric manner. Finally, we provide some promising future directions for data-centric graph learning.
人工智能（AI）的历史见证了高质量数据对各种深度学习模型（如ImageNet for AlexNet和ResNet）的重大影响。最近，人工智能界的注意力从设计更复杂的神经架构作为以模型为中心的方法，转向以数据为中心的方法，即更好地处理数据以增强神经模型的能力。在本调查中，我们从数据中心的角度全面回顾了图学习方法，并旨在回答三个关键问题：(1)何时修改图数据，(2)需要修改图数据的哪一部分以释放各种图模型的潜力，以及(3)如何保护图模型免受问题数据的影响。因此，我们提出了一种基于图学习管道阶段的新分类方法，并重点介绍了图数据中不同数据结构（拓扑、特征和标签）的处理方法。此外，我们还分析了图数据中的一些潜在问题，并讨论了如何以数据为中心的方式解决这些问题。最后，我们为以数据为中心的图学习提供了一些有希望的未来方向。
{"title":"Data-Centric Graph Learning: A Survey","authors":"Yuxin Guo;Deyu Bo;Cheng Yang;Zhiyuan Lu;Zhongjian Zhang;Jixi Liu;Yufei Peng;Chuan Shi","doi":"10.1109/TBDATA.2024.3489412","DOIUrl":"https://doi.org/10.1109/TBDATA.2024.3489412","url":null,"abstract":"The history of artificial intelligence (AI) has witnessed the significant impact of high-quality data on various deep learning models, such as ImageNet for AlexNet and ResNet. Recently, instead of designing more complex neural architectures as model-centric approaches, the attention of AI community has shifted to data-centric ones, which focuses on better processing data to strengthen the ability of neural models. Graph learning, which operates on ubiquitous topological data, also plays an important role in the era of deep learning. In this survey, we comprehensively review graph learning approaches from the data-centric perspective, and aim to answer three crucial questions: <italic>(1) when to modify graph data</i>, <italic>(2) what part of the graph data needs modification</i> to unlock the potential of various graph models, and <italic>(3) how to safeguard graph models</i> from problematic data influence. Accordingly, we propose a novel taxonomy based on the stages in the graph learning pipeline, and highlight the processing methods for different data structures in the graph data, i.e., topology, feature and label. Furthermore, we analyze some potential problems embedded in graph data and discuss how to solve them in a data-centric manner. Finally, we provide some promising future directions for data-centric graph learning.","PeriodicalId":13106,"journal":{"name":"IEEE Transactions on Big Data","volume":"11 1","pages":"1-20"},"PeriodicalIF":7.5,"publicationDate":"2024-11-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142993790","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-01DOI: 10.1109/TBDATA.2024.3489419
Qihang Wang;Ying Chen;Qinge Xiao
To determine the real-time changes in brain arousal introduced by anesthetics, Electroencephalogram (EEG) is often used as an objective neuroimaging evidence to link the neurobehavioral states of patients. However, EEG signals often suffer from a low signal-to-noise ratio due to environmental noise and artifacts, which limits its application for a reliable estimation of depth of anesthesia (DoA), especially under high cross-subject variability. In this study, we propose an end-to-end deep learning based framework, termed as AnesFormer, which contains a data selection model, a self-attention based classification model, and a baseline update mechanism. These three components are integrated in a dynamic and seamless manner to achieve the goal of improving the effectiveness and robustness of DoA estimation in a leave-one-out setting. In the experiment, we apply the proposed framework to an office-based dataset and a hospital-based dataset, and use seven existing models as benchmarks. In addition, we conduct an ablation experiment to show the significance of each component in AnesFormer. Our main results indicate that 1) the proposed framework generally performs better than the existing methods for DoA estimation in terms of effectiveness and robustness; 2) each designed component in AnesFormer is likely to contribute to the DoA classification improvement.
{"title":"AnesFormer: An End-to-End Framework for EEG-Based Anesthetic State Classification","authors":"Qihang Wang;Ying Chen;Qinge Xiao","doi":"10.1109/TBDATA.2024.3489419","DOIUrl":"https://doi.org/10.1109/TBDATA.2024.3489419","url":null,"abstract":"To determine the real-time changes in brain arousal introduced by anesthetics, Electroencephalogram (EEG) is often used as an objective neuroimaging evidence to link the neurobehavioral states of patients. However, EEG signals often suffer from a low signal-to-noise ratio due to environmental noise and artifacts, which limits its application for a reliable estimation of depth of anesthesia (DoA), especially under high cross-subject variability. In this study, we propose an end-to-end deep learning based framework, termed as AnesFormer, which contains a data selection model, a self-attention based classification model, and a baseline update mechanism. These three components are integrated in a dynamic and seamless manner to achieve the goal of improving the effectiveness and robustness of DoA estimation in a leave-one-out setting. In the experiment, we apply the proposed framework to an office-based dataset and a hospital-based dataset, and use seven existing models as benchmarks. In addition, we conduct an ablation experiment to show the significance of each component in AnesFormer. Our main results indicate that 1) the proposed framework generally performs better than the existing methods for DoA estimation in terms of effectiveness and robustness; 2) each designed component in AnesFormer is likely to contribute to the DoA classification improvement.","PeriodicalId":13106,"journal":{"name":"IEEE Transactions on Big Data","volume":"11 3","pages":"1357-1368"},"PeriodicalIF":7.5,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143949175","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Stable learning aims to leverage the knowledge in a relevant source domain to learn a prediction model that can generalize well to target domains. Recent advances in stable learning mainly proceed by eliminating spurious correlations between irrelevant features and labels through sample reweighting or causal feature selection. However, most existing stable learning methods either only weaken partial spurious correlations or discard part of true causal relationships, resulting in generalization performance degradation. To tackle these issues, we propose the Dual Feature Learning (DFL) algorithm for stable learning, which consists of two phases. Phase 1 first learns a set of sample weights to balance the distribution of treated and control groups corresponding to each feature, and then uses the learned sample weights to assist feature selection to identify part of irrelevant features for completely isolating spurious correlations between these irrelevant features and labels. Phase 2 first learns two groups of sample weights again using the subdataset after feature selection, and then obtains high-quality feature representations by integrating a weighted cross-entropy model and an autoencoder model to further get rid of spurious correlations. Using synthetic and four real-world datasets, the experiments have verified the effectiveness of DFL, in comparison with eleven state-of-the-art methods.
{"title":"Stable Learning via Dual Feature Learning","authors":"Shuai Yang;Xin Li;Minzhi Wu;Qianlong Dang;Lichuan Gu","doi":"10.1109/TBDATA.2024.3489413","DOIUrl":"https://doi.org/10.1109/TBDATA.2024.3489413","url":null,"abstract":"Stable learning aims to leverage the knowledge in a relevant source domain to learn a prediction model that can generalize well to target domains. Recent advances in stable learning mainly proceed by eliminating spurious correlations between irrelevant features and labels through sample reweighting or causal feature selection. However, most existing stable learning methods either only weaken partial spurious correlations or discard part of true causal relationships, resulting in generalization performance degradation. To tackle these issues, we propose the Dual Feature Learning (DFL) algorithm for stable learning, which consists of two phases. Phase 1 first learns a set of sample weights to balance the distribution of treated and control groups corresponding to each feature, and then uses the learned sample weights to assist feature selection to identify part of irrelevant features for completely isolating spurious correlations between these irrelevant features and labels. Phase 2 first learns two groups of sample weights again using the subdataset after feature selection, and then obtains high-quality feature representations by integrating a weighted cross-entropy model and an autoencoder model to further get rid of spurious correlations. Using synthetic and four real-world datasets, the experiments have verified the effectiveness of DFL, in comparison with eleven state-of-the-art methods.","PeriodicalId":13106,"journal":{"name":"IEEE Transactions on Big Data","volume":"11 4","pages":"1852-1866"},"PeriodicalIF":7.5,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144597667","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In recent years, the Graph Transformer has demonstrated superiority on various graph-level tasks by facilitating global interactions among nodes. However, as for node-level tasks, the existing Graph Transformer cannot perform as well as expected. Actually, a node in a real-world graph does not necessarily have relationships with every other node, and this global interaction weakens node features. This raises a fundamental question: should we partition out an appropriate interaction channel based on graph structure so that noisy and irrelevant information will be filtered and every node can aggregate information in the optimal channel? We first perform a series of experiments on manually created graphs with varying homophily ratios. Surprisingly, we observe that different graph structures indeed require distinct optimal interaction channels. This leads us to ask whether we can develop a partitioning rule that ensures each node interacts with relevant and valuable targets. To overcome this challenge, we propose a novel Graph Transformer named Multi-channel Graphormer. The model is evaluated on six network datasets with different homophily ratios for the node classification task. Moreover, comprehensive experiments are conducted on two real datasets for the recommendation task. Experimental results show that the Multi-channel Graphormer surpasses state-of-the-art baselines, demonstrating superior performance.
{"title":"M-Graphormer: Multi-Channel Graph Transformer for Node Representation Learning","authors":"Xinglong Chang;Jianrong Wang;Mingxiang Wen;Yingkui Wang;Yuxiao Huang","doi":"10.1109/TBDATA.2024.3489418","DOIUrl":"https://doi.org/10.1109/TBDATA.2024.3489418","url":null,"abstract":"In recent years, the Graph Transformer has demonstrated superiority on various graph-level tasks by facilitating global interactions among nodes. However, as for node-level tasks, the existing Graph Transformer cannot perform as well as expected. Actually, a node in a real-world graph does not necessarily have relationships with every other node, and this global interaction weakens node features. This raises a fundamental question: should we partition out an appropriate interaction channel based on graph structure so that noisy and irrelevant information will be filtered and every node can aggregate information in the optimal channel? We first perform a series of experiments on manually created graphs with varying homophily ratios. Surprisingly, we observe that different graph structures indeed require distinct optimal interaction channels. This leads us to ask whether we can develop a partitioning rule that ensures each node interacts with relevant and valuable targets. To overcome this challenge, we propose a novel Graph Transformer named Multi-channel Graphormer. The model is evaluated on six network datasets with different homophily ratios for the node classification task. Moreover, comprehensive experiments are conducted on two real datasets for the recommendation task. Experimental results show that the Multi-channel Graphormer surpasses state-of-the-art baselines, demonstrating superior performance.","PeriodicalId":13106,"journal":{"name":"IEEE Transactions on Big Data","volume":"11 4","pages":"1867-1878"},"PeriodicalIF":7.5,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144597811","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-11-01DOI: 10.1109/TBDATA.2024.3489414
Altaf Hussain;Tanveer Hussain;Waseem Ullah;Samee Ullah Khan;Min Je Kim;Khan Muhammad;Javier Del Ser;Sung Wook Baik
Deep-learning-based human activity recognition (HAR) methods have significantly transformed a wide range of domains over recent years. However, the adoption of Big Data techniques in industrial applications remains challenging due to issues such as generalized weight optimization, diverse viewpoints, and the complex spatiotemporal features of videos. To address these challenges, this work presents an industrial HAR framework consisting of two main phases. First, a squeeze bottleneck attention block (SBAB) is introduced to enhance the learning capabilities of the backbone model for contextual learning, which allows for the selection and refinement of an optimal feature vector. In the second phase, we propose an effective sequential temporal convolutional network (STCN), which is designed in parallel fashion to mitigate the issues of exploding and vanishing gradients associated with sequence learning. The high-dimensional spatiotemporal feature vectors from the STCN undergo further refinement through our proposed SBAB in a sequential manner, to optimize the features for HAR and enhance the overall performance. The efficacy of the proposed framework is validated through extensive experiments on six datasets, including data from industrial and general activities.
{"title":"Big Data Analysis for Industrial Activity Recognition Using Attention-Inspired Sequential Temporal Convolution Network","authors":"Altaf Hussain;Tanveer Hussain;Waseem Ullah;Samee Ullah Khan;Min Je Kim;Khan Muhammad;Javier Del Ser;Sung Wook Baik","doi":"10.1109/TBDATA.2024.3489414","DOIUrl":"https://doi.org/10.1109/TBDATA.2024.3489414","url":null,"abstract":"Deep-learning-based human activity recognition (HAR) methods have significantly transformed a wide range of domains over recent years. However, the adoption of Big Data techniques in industrial applications remains challenging due to issues such as generalized weight optimization, diverse viewpoints, and the complex spatiotemporal features of videos. To address these challenges, this work presents an industrial HAR framework consisting of two main phases. First, a squeeze bottleneck attention block (SBAB) is introduced to enhance the learning capabilities of the backbone model for contextual learning, which allows for the selection and refinement of an optimal feature vector. In the second phase, we propose an effective sequential temporal convolutional network (STCN), which is designed in parallel fashion to mitigate the issues of exploding and vanishing gradients associated with sequence learning. The high-dimensional spatiotemporal feature vectors from the STCN undergo further refinement through our proposed SBAB in a sequential manner, to optimize the features for HAR and enhance the overall performance. The efficacy of the proposed framework is validated through extensive experiments on six datasets, including data from industrial and general activities.","PeriodicalId":13106,"journal":{"name":"IEEE Transactions on Big Data","volume":"11 4","pages":"1840-1851"},"PeriodicalIF":7.5,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144597668","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Fingerprint localization methods typically require a substantial amount of manual effort to collect fingerprint data from various scenarios to construct an accurate radio map. While some existing research has attempted to use path planning strategies to save on labor costs, these approaches often suffer from being time-consuming and prone to locally optimal solutions. To address these shortcomings, our paper proposes a novel approach that utilizes imitation learning to construct and update a highly accurate radio map with minimal manual intervention in dynamic environments. Specifically, we employ a multivariate Gaussian process model to fit a rough standby fingerprint database with only a few pilot data points. We then utilize a state space model to calculate the variation range of the pilot data, which forms the CSI error band used to filter the rough radio map. Imitation learning and a confidence coefficient are utilized to predict and calibrate the global CSI data distribution. And we utilize the K-nearest neighbor algorithm to achieve the real-time localization function. Experimental results show that our proposed algorithm outperforms several state-of-the-art approaches in most test cases, exhibiting low computation complexity, lower localization error, and saving 73.3% of the manual workload.
{"title":"Dynamic Radio Map Construction With Minimal Manual Intervention: A State Space Model-Based Approach With Imitation Learning","authors":"Xiaoqiang Zhu;Tie Qiu;Wenyu Qu;Xiaobo Zhou;Tuo Shi;Tianyi Xu","doi":"10.1109/TBDATA.2024.3489425","DOIUrl":"https://doi.org/10.1109/TBDATA.2024.3489425","url":null,"abstract":"Fingerprint localization methods typically require a substantial amount of manual effort to collect fingerprint data from various scenarios to construct an accurate radio map. While some existing research has attempted to use path planning strategies to save on labor costs, these approaches often suffer from being time-consuming and prone to locally optimal solutions. To address these shortcomings, our paper proposes a novel approach that utilizes imitation learning to construct and update a highly accurate radio map with minimal manual intervention in dynamic environments. Specifically, we employ a multivariate Gaussian process model to fit a rough standby fingerprint database with only a few pilot data points. We then utilize a state space model to calculate the variation range of the pilot data, which forms the CSI error band used to filter the rough radio map. Imitation learning and a confidence coefficient are utilized to predict and calibrate the global CSI data distribution. And we utilize the K-nearest neighbor algorithm to achieve the real-time localization function. Experimental results show that our proposed algorithm outperforms several state-of-the-art approaches in most test cases, exhibiting low computation complexity, lower localization error, and saving 73.3% of the manual workload.","PeriodicalId":13106,"journal":{"name":"IEEE Transactions on Big Data","volume":"11 4","pages":"1799-1812"},"PeriodicalIF":7.5,"publicationDate":"2024-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144597828","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-31DOI: 10.1109/TBDATA.2024.3489421
Luyi Bai;Linshuo Xu;Lin Zhu
Answering complex logical queries on large-scale Knowledge Graphs (KGs) efficiently and accurately has always been crucial for question-answering systems. Recent studies have significantly improved the performance of complex logical queries on massive knowledge graphs by leveraging graph neural networks (GNNs). However, the existing GNN-based methods still have limitations in dealing with long-sequence logical queries. They usually decompose complex queries into multiple independent first-order logical queries, which leads to the inability to optimize globally, and the query accuracy will drop sharply with the increase of query length. In addition, the knowlege in the real world is dynamically changing, but most of the existing methods are more suitable for dealing with static knowledge graphs, and there is still much room for improvement when dealing with logical queries in temporal knowledge graphs. In this paper, we propose a novel Temporal Complex Logical Query (TCLQ) model to achieve temporal logical queries on temporal knowledge graphs. We add time series embedding into GNN, and use multi-layer GRUs to aggregate the node features of previous time and current time, which effectively enhances the time series reasoning ability of the model. In order to solve the problem that the accuracy of logical query model decreases significantly with the increase of query sequence length, we establish a multi-level attention coefficients model to learn and optimize the whole logical queries, thus reducing the error accumulation problem when the queries are decomposed into multiple independent first-order logical queries. We conduct experiments on multiple temporal datasets and demonstrate the effectiveness of TCLQ.
{"title":"Attention-Based Complex Logical Query on Temporal Knowledge Graph via Graph Neural Network","authors":"Luyi Bai;Linshuo Xu;Lin Zhu","doi":"10.1109/TBDATA.2024.3489421","DOIUrl":"https://doi.org/10.1109/TBDATA.2024.3489421","url":null,"abstract":"Answering complex logical queries on large-scale Knowledge Graphs (KGs) efficiently and accurately has always been crucial for question-answering systems. Recent studies have significantly improved the performance of complex logical queries on massive knowledge graphs by leveraging graph neural networks (GNNs). However, the existing GNN-based methods still have limitations in dealing with long-sequence logical queries. They usually decompose complex queries into multiple independent first-order logical queries, which leads to the inability to optimize globally, and the query accuracy will drop sharply with the increase of query length. In addition, the knowlege in the real world is dynamically changing, but most of the existing methods are more suitable for dealing with static knowledge graphs, and there is still much room for improvement when dealing with logical queries in temporal knowledge graphs. In this paper, we propose a novel Temporal Complex Logical Query (TCLQ) model to achieve temporal logical queries on temporal knowledge graphs. We add time series embedding into GNN, and use multi-layer GRUs to aggregate the node features of previous time and current time, which effectively enhances the time series reasoning ability of the model. In order to solve the problem that the accuracy of logical query model decreases significantly with the increase of query sequence length, we establish a multi-level attention coefficients model to learn and optimize the whole logical queries, thus reducing the error accumulation problem when the queries are decomposed into multiple independent first-order logical queries. We conduct experiments on multiple temporal datasets and demonstrate the effectiveness of TCLQ.","PeriodicalId":13106,"journal":{"name":"IEEE Transactions on Big Data","volume":"11 4","pages":"1828-1839"},"PeriodicalIF":7.5,"publicationDate":"2024-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144597729","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Graph neural networks (GNNs) exhibit a robust capability for representation learning on graphs with complex structures, demonstrating superior performance across various applications. Most existing GNNs utilize graph convolution operations that integrate both attribute and structural information through coupled way. And these GNNs, from an optimization perspective, seek to learn a consensus and compromised embedding representation that balances attribute and graph information, selectively exploring and retaining valid information in essence. To obtain a more comprehensive embedding representation, a novel GNN framework, dubbed Decoupled Graph Neural Networks (DGNN), is introduced. DGNN separately explores distinctive embedding representations from the attribute and graph spaces by decoupled terms. Considering that the semantic graph, derived from attribute feature space, contains different node connection information and provides enhancement for the topological graph, both topological and semantic graphs are integrated by DGNN for powerful embedding representation learning. Further, structural consistency between the attribute embedding and the graph embedding is promoted to effectively eliminate redundant information and establish soft connection. This process involves facilitating factor sharing for adjacency matrices reconstruction, which aims at exploring consensus and high-level correlations. Finally, a more powerful and comprehensive representation is achieved through the concatenation of these embeddings. Experimental results conducted on several graph benchmark datasets demonstrate its superiority in node classification tasks.
{"title":"DGNN: Decoupled Graph Neural Networks With Structural Consistency Between Attribute and Graph Embedding Representations","authors":"Jinlu Wang;Jipeng Guo;Yanfeng Sun;Junbin Gao;Shaofan Wang;Yachao Yang;Baocai Yin","doi":"10.1109/TBDATA.2024.3489420","DOIUrl":"https://doi.org/10.1109/TBDATA.2024.3489420","url":null,"abstract":"Graph neural networks (GNNs) exhibit a robust capability for representation learning on graphs with complex structures, demonstrating superior performance across various applications. Most existing GNNs utilize graph convolution operations that integrate both attribute and structural information through coupled way. And these GNNs, from an optimization perspective, seek to learn a consensus and compromised embedding representation that balances attribute and graph information, selectively exploring and retaining valid information in essence. To obtain a more comprehensive embedding representation, a novel GNN framework, dubbed Decoupled Graph Neural Networks (DGNN), is introduced. DGNN separately explores distinctive embedding representations from the attribute and graph spaces by decoupled terms. Considering that the semantic graph, derived from attribute feature space, contains different node connection information and provides enhancement for the topological graph, both topological and semantic graphs are integrated by DGNN for powerful embedding representation learning. Further, structural consistency between the attribute embedding and the graph embedding is promoted to effectively eliminate redundant information and establish soft connection. This process involves facilitating factor sharing for adjacency matrices reconstruction, which aims at exploring consensus and high-level correlations. Finally, a more powerful and comprehensive representation is achieved through the concatenation of these embeddings. Experimental results conducted on several graph benchmark datasets demonstrate its superiority in node classification tasks.","PeriodicalId":13106,"journal":{"name":"IEEE Transactions on Big Data","volume":"11 4","pages":"1813-1827"},"PeriodicalIF":7.5,"publicationDate":"2024-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144597808","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-31DOI: 10.1109/TBDATA.2024.3489415
Jianghui Sang;Yongli Wang;Zaki Ahmad Khan;Xiaoliang Zhou
Existing research on potential-based reward shaping (PBRS) relies on optimal policy in Markov decision process (MDP) where optimal policy is regarded as the ground truth. However, in some practical application scenarios, there is an extrapolation error challenge between the computed optimal policy and the real-world optimal policy. At this time, the optimal policy is unreliable. To address this challenge, we design a Reward Shaping based on Optimal-Policy-Free to get rid of the dependence on the optimal policy. We view reinforcement learning as probabilistic inference on a directed graph. Essentially, this inference propagates information from the rewarding states in the MDP and results in a function which is leveraged as a potential function for PBRS. Our approach utilizes a contrastive learning technique on directed graph Laplacian. Here, this technique does not change the structure of the directed graph. Then, the directed graph Laplacian is used to approximate the true state transition matrix in MDP. The potential function in PBRS can be learned through the message passing mechanism which is built on this directed graph Laplacian. The experiments on Atari, MuJoCo and MiniWorld show that our approach outperforms the competitive algorithms.
{"title":"Reward Shaping Based on Optimal-Policy-Free","authors":"Jianghui Sang;Yongli Wang;Zaki Ahmad Khan;Xiaoliang Zhou","doi":"10.1109/TBDATA.2024.3489415","DOIUrl":"https://doi.org/10.1109/TBDATA.2024.3489415","url":null,"abstract":"Existing research on potential-based reward shaping (PBRS) relies on optimal policy in Markov decision process (MDP) where optimal policy is regarded as the ground truth. However, in some practical application scenarios, there is an extrapolation error challenge between the computed optimal policy and the real-world optimal policy. At this time, the optimal policy is unreliable. To address this challenge, we design a Reward Shaping based on Optimal-Policy-Free to get rid of the dependence on the optimal policy. We view reinforcement learning as probabilistic inference on a directed graph. Essentially, this inference propagates information from the rewarding states in the MDP and results in a function which is leveraged as a potential function for PBRS. Our approach utilizes a contrastive learning technique on directed graph Laplacian. Here, this technique does not change the structure of the directed graph. Then, the directed graph Laplacian is used to approximate the true state transition matrix in MDP. The potential function in PBRS can be learned through the message passing mechanism which is built on this directed graph Laplacian. The experiments on Atari, MuJoCo and MiniWorld show that our approach outperforms the competitive algorithms.","PeriodicalId":13106,"journal":{"name":"IEEE Transactions on Big Data","volume":"11 4","pages":"1787-1798"},"PeriodicalIF":7.5,"publicationDate":"2024-10-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144597666","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：