Pub Date : 2024-09-30DOI: 10.1109/TKDE.2024.3471659
Aoting Zeng;Liping Wang;Wenjie Zhang;Xuemin Lin
Graph Neural Networks (GNNs) with data augmentation obtain promising results among existing solutions for graph classification. Mixup-based augmentation methods for graph classification have already achieved state-of-the-art performance. However, existing mixup-based augmentation methods either operate in the input space and thus face the challenge of balancing efficiency and accuracy, or directly conduct mixup in the latent space without similarity guarantee, thus leading to lacking semantic validity and limited performance. To address these limitations, this paper proposes $mathcal {G}$ -MixCon, a novel framework leveraging the strengths of Mix up-based augmentation and supervised Con trastive learning (SCL). To the best of our knowledge, this is the first attempt to develop an SCL-based approach for learning graph representations. Specifically, the mixup-based strategy within the latent space named $GDA_{gl}$ and $GDA_{nl}$ are proposed, which efficiently conduct linear interpolation between views of the node or graph level. Furthermore, we design a dual-objective loss function named SupMixCon that can consider both the consistency among graphs and the distances between the original and augmented graph. SupMixCon can guide the training process for SCL in $mathcal {G}$ -MixCon while achieving a similarity guarantee. Comprehensive experiments are conducted on various real-world datasets, the results show that $mathcal {G}$ -MixCon demonstrably enhances performance, achieving an average accuracy increment of 6.24%, and significantly increases the robustness of GNNs against noisy labels.
{"title":"Efficient and Effective Augmentation Framework With Latent Mixup and Label-Guided Contrastive Learning for Graph Classification","authors":"Aoting Zeng;Liping Wang;Wenjie Zhang;Xuemin Lin","doi":"10.1109/TKDE.2024.3471659","DOIUrl":"https://doi.org/10.1109/TKDE.2024.3471659","url":null,"abstract":"Graph Neural Networks (GNNs) with data augmentation obtain promising results among existing solutions for graph classification. Mixup-based augmentation methods for graph classification have already achieved state-of-the-art performance. However, existing mixup-based augmentation methods either operate in the input space and thus face the challenge of balancing efficiency and accuracy, or directly conduct mixup in the latent space without similarity guarantee, thus leading to lacking semantic validity and limited performance. To address these limitations, this paper proposes \u0000<inline-formula><tex-math>$mathcal {G}$</tex-math></inline-formula>\u0000-MixCon, a novel framework leveraging the strengths of \u0000<i><u>Mix</u></i>\u0000up-based augmentation and supervised \u0000<i><u>Con</u></i>\u0000trastive learning (SCL). To the best of our knowledge, this is the first attempt to develop an SCL-based approach for learning graph representations. Specifically, the mixup-based strategy within the latent space named \u0000<inline-formula><tex-math>$GDA_{gl}$</tex-math></inline-formula>\u0000 and \u0000<inline-formula><tex-math>$GDA_{nl}$</tex-math></inline-formula>\u0000 are proposed, which efficiently conduct linear interpolation between views of the node or graph level. Furthermore, we design a dual-objective loss function named \u0000<i>SupMixCon</i>\u0000 that can consider both the consistency among graphs and the distances between the original and augmented graph. \u0000<i>SupMixCon</i>\u0000 can guide the training process for SCL in \u0000<inline-formula><tex-math>$mathcal {G}$</tex-math></inline-formula>\u0000-MixCon while achieving a similarity guarantee. Comprehensive experiments are conducted on various real-world datasets, the results show that \u0000<inline-formula><tex-math>$mathcal {G}$</tex-math></inline-formula>\u0000-MixCon demonstrably enhances performance, achieving an average accuracy increment of 6.24%, and significantly increases the robustness of GNNs against noisy labels.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"36 12","pages":"8066-8078"},"PeriodicalIF":8.9,"publicationDate":"2024-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636574","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-27DOI: 10.1109/TKDE.2024.3469578
Bowen Jin;Gang Liu;Chi Han;Meng Jiang;Heng Ji;Jiawei Han
Large language models (LLMs), such as GPT4 and LLaMA, are creating significant advancements in natural language processing, due to their strong text encoding/decoding ability and newly found emergent capability (e.g., reasoning). While LLMs are mainly designed to process pure texts, there are many real-world scenarios where text data is associated with rich structure information in the form of graphs (e.g., academic networks, and e-commerce networks) or scenarios where graph data is paired with rich textual information (e.g., molecules with descriptions). Besides, although LLMs have shown their pure text-based reasoning ability, it is underexplored whether such ability can be generalized to graphs (i.e., graph-based reasoning). In this paper, we provide a systematic review of scenarios and techniques related to large language models on graphs. We first summarize potential scenarios of adopting LLMs on graphs into three categories, namely pure graphs, text-attributed graphs, and text-paired graphs. We then discuss detailed techniques for utilizing LLMs on graphs, including LLM as Predictor, LLM as Encoder, and LLM as Aligner, and compare the advantages and disadvantages of different schools of models. Furthermore, we discuss the real-world applications of such methods and summarize open-source codes and benchmark datasets. Finally, we conclude with potential future research directions in this fast-growing field.
{"title":"Large Language Models on Graphs: A Comprehensive Survey","authors":"Bowen Jin;Gang Liu;Chi Han;Meng Jiang;Heng Ji;Jiawei Han","doi":"10.1109/TKDE.2024.3469578","DOIUrl":"https://doi.org/10.1109/TKDE.2024.3469578","url":null,"abstract":"Large language models (LLMs), such as GPT4 and LLaMA, are creating significant advancements in natural language processing, due to their strong text encoding/decoding ability and newly found emergent capability (e.g., reasoning). While LLMs are mainly designed to process pure texts, there are many real-world scenarios where text data is associated with rich structure information in the form of graphs (e.g., academic networks, and e-commerce networks) or scenarios where graph data is paired with rich textual information (e.g., molecules with descriptions). Besides, although LLMs have shown their pure text-based reasoning ability, it is underexplored whether such ability can be generalized to graphs (i.e., graph-based reasoning). In this paper, we provide a systematic review of scenarios and techniques related to large language models on graphs. We first summarize potential scenarios of adopting LLMs on graphs into three categories, namely pure graphs, text-attributed graphs, and text-paired graphs. We then discuss detailed techniques for utilizing LLMs on graphs, including LLM as Predictor, LLM as Encoder, and LLM as Aligner, and compare the advantages and disadvantages of different schools of models. Furthermore, we discuss the real-world applications of such methods and summarize open-source codes and benchmark datasets. Finally, we conclude with potential future research directions in this fast-growing field.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"36 12","pages":"8622-8642"},"PeriodicalIF":8.9,"publicationDate":"2024-09-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142645400","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-27DOI: 10.1109/TKDE.2024.3469782
Qianxin Yi;Yiyang Yang;Shaojie Tang;Jiapeng Liu;Yao Wang
In this paper, we aim to build a novel bandits algorithm that is capable of fully harnessing the power of multi-dimensional data and the inherent non-linearity of reward functions to provide high-usable and accountable decision-making services. To this end, we introduce a generalized low-rank tensor contextual bandits model in which an action is formed from three feature vectors, and thus is represented by a tensor. In this formulation, the reward is determined through a generalized linear function applied to the inner product of the action’s feature tensor and a fixed but unknown parameter tensor with low-rank structure. To effectively achieve the trade-off between exploration and exploitation, we introduce an algorithm called “Generalized Low-Rank Tensor Exploration Subspace then Refine” (G-LowTESTR). This algorithm first collects data to explore the intrinsic low-rank tensor subspace information embedded in the scenario, and then converts the original problem into a lower-dimensional generalized linear contextual bandits problem. Rigorous theoretical analysis shows that the regret bound of G-LowTESTR is superior to those in vectorization and matricization cases. We conduct a series of synthetic and real data experiments to further highlight the effectiveness of G-LowTESTR, leveraging its ability to capitalize on the low-rank tensor structure for enhanced learning.
{"title":"Effective Generalized Low-Rank Tensor Contextual Bandits","authors":"Qianxin Yi;Yiyang Yang;Shaojie Tang;Jiapeng Liu;Yao Wang","doi":"10.1109/TKDE.2024.3469782","DOIUrl":"https://doi.org/10.1109/TKDE.2024.3469782","url":null,"abstract":"In this paper, we aim to build a novel bandits algorithm that is capable of fully harnessing the power of multi-dimensional data and the inherent non-linearity of reward functions to provide high-usable and accountable decision-making services. To this end, we introduce a generalized low-rank tensor contextual bandits model in which an action is formed from three feature vectors, and thus is represented by a tensor. In this formulation, the reward is determined through a generalized linear function applied to the inner product of the action’s feature tensor and a fixed but unknown parameter tensor with low-rank structure. To effectively achieve the trade-off between exploration and exploitation, we introduce an algorithm called “Generalized Low-Rank Tensor Exploration Subspace then Refine” (G-LowTESTR). This algorithm first collects data to explore the intrinsic low-rank tensor subspace information embedded in the scenario, and then converts the original problem into a lower-dimensional generalized linear contextual bandits problem. Rigorous theoretical analysis shows that the regret bound of G-LowTESTR is superior to those in vectorization and matricization cases. We conduct a series of synthetic and real data experiments to further highlight the effectiveness of G-LowTESTR, leveraging its ability to capitalize on the low-rank tensor structure for enhanced learning.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"36 12","pages":"8051-8065"},"PeriodicalIF":8.9,"publicationDate":"2024-09-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636476","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Graph Contrastive Learning (GCL) stands as a potent framework for unsupervised graph representation learning that has gained traction across numerous graph learning applications. The effectiveness of GCL relies on generating high-quality contrasting samples, enhancing the model’s ability to discern graph semantics. However, the prevailing GCL methods face two key challenges: 1) introducing noise during graph augmentations and 2) requiring additional storage for generated samples, which degrade the model performance. In this paper, we propose novel approaches, GKCL (i.e., Graph Knowledge Contrastive Learning) and DGKCL (i.e., Distilled Graph Knowledge Contrastive Learning), that leverage multi-level graph knowledge to create noise-free contrasting pairs. This framework not only addresses the noise-related challenges but also circumvents excessive storage demands. Furthermore, our method incorporates a knowledge distillation component to optimize the trained embedding tables, reducing the model’s scale while ensuring superior performance, particularly for the scenarios with smaller embedding sizes. Comprehensive experimental evaluations on three public benchmark datasets underscore the merits of our proposed method and elucidate its properties, which primarily reflect the performance of the proposed method equipped with different embedding sizes and how the distillation weight affects the overall performance.
{"title":"Multi-Level Graph Knowledge Contrastive Learning","authors":"Haoran Yang;Yuhao Wang;Xiangyu Zhao;Hongxu Chen;Hongzhi Yin;Qing Li;Guandong Xu","doi":"10.1109/TKDE.2024.3466530","DOIUrl":"https://doi.org/10.1109/TKDE.2024.3466530","url":null,"abstract":"Graph Contrastive Learning (GCL) stands as a potent framework for unsupervised graph representation learning that has gained traction across numerous graph learning applications. The effectiveness of GCL relies on generating high-quality contrasting samples, enhancing the model’s ability to discern graph semantics. However, the prevailing GCL methods face two key challenges: 1) introducing noise during graph augmentations and 2) requiring additional storage for generated samples, which degrade the model performance. In this paper, we propose novel approaches, GKCL (i.e., Graph Knowledge Contrastive Learning) and DGKCL (i.e., Distilled Graph Knowledge Contrastive Learning), that leverage multi-level graph knowledge to create noise-free contrasting pairs. This framework not only addresses the noise-related challenges but also circumvents excessive storage demands. Furthermore, our method incorporates a knowledge distillation component to optimize the trained embedding tables, reducing the model’s scale while ensuring superior performance, particularly for the scenarios with smaller embedding sizes. Comprehensive experimental evaluations on three public benchmark datasets underscore the merits of our proposed method and elucidate its properties, which primarily reflect the performance of the proposed method equipped with different embedding sizes and how the distillation weight affects the overall performance.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"36 12","pages":"8829-8841"},"PeriodicalIF":8.9,"publicationDate":"2024-09-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636365","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-26DOI: 10.1109/TKDE.2024.3464516
Lingyuan Meng;Ke Liang;Hao Yu;Yue Liu;Sihang Zhou;Meng Liu;Xinwang Liu
Federated knowledge graph reasoning (FedKGR) aims to perform reasoning over different clients while protecting data privacy, drawing increasing attention to its high practical value. Previous works primarily focus on data heterogeneity, ignoring challenges from limited data scale and primitive negative sample strategies, i.e., random entity replacement, which yield low-quality negatives and zero loss issues. Meanwhile, generative adversarial networks (GANs) are widely used in different fields to generate high-quality negative samples, but no work has been developed for FedKGR. To this end, we propose a plug-and-play E ntity-aware A dversarial N egative sampling strategy for FedKGR, termed FedEAN. Specifically, we are the first to adopt GANs to generate high-quality negative samples in different clients. It takes the target triplet in each batch as input and outputs high-quality negative samples, which guaranteed by the joint training of the generator and discriminator. Moreover, we design an entity-aware adaptive negative sampling mechanism based on the similarity of entity representations before and after server aggregation, which can persevere the entity global consistency across clients during training. Extensive experiments demonstrate that FedEAN excels with various FedKGR backbones, demonstrating its ability to construct high-quality negative samples and address the zero-loss issue.
{"title":"FedEAN: Entity-Aware Adversarial Negative Sampling for Federated Knowledge Graph Reasoning","authors":"Lingyuan Meng;Ke Liang;Hao Yu;Yue Liu;Sihang Zhou;Meng Liu;Xinwang Liu","doi":"10.1109/TKDE.2024.3464516","DOIUrl":"https://doi.org/10.1109/TKDE.2024.3464516","url":null,"abstract":"Federated knowledge graph reasoning (FedKGR) aims to perform reasoning over different clients while protecting data privacy, drawing increasing attention to its high practical value. Previous works primarily focus on data heterogeneity, ignoring challenges from limited data scale and primitive negative sample strategies, i.e., random entity replacement, which yield low-quality negatives and zero loss issues. Meanwhile, generative adversarial networks (GANs) are widely used in different fields to generate high-quality negative samples, but no work has been developed for FedKGR. To this end, we propose a plug-and-play \u0000<underline>E</u>\u0000ntity-aware \u0000<underline>A</u>\u0000dversarial \u0000<underline>N</u>\u0000egative sampling strategy for FedKGR, termed FedEAN. Specifically, we are the first to adopt GANs to generate high-quality negative samples in different clients. It takes the target triplet in each batch as input and outputs high-quality negative samples, which guaranteed by the joint training of the generator and discriminator. Moreover, we design an entity-aware adaptive negative sampling mechanism based on the similarity of entity representations before and after server aggregation, which can persevere the entity global consistency across clients during training. Extensive experiments demonstrate that FedEAN excels with various FedKGR backbones, demonstrating its ability to construct high-quality negative samples and address the zero-loss issue.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"36 12","pages":"8206-8219"},"PeriodicalIF":8.9,"publicationDate":"2024-09-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600232","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-25DOI: 10.1109/TKDE.2024.3466990
Youming Liu;Lin Shu;Chuan Chen;Zibin Zheng
Graph contrastive learning defines a contrastive task to pull similar instances close and push dissimilar instances away. It learns discriminative node embeddings without supervised labels, which has aroused increasing attention in the past few years. Nevertheless, existing methods of graph contrastive learning ignore the differences between diverse semantics existed in graphs, which learn coarse-grained node embeddings and lead to sub-optimal performances on downstream tasks. To bridge this gap, we propose a novel F ine-grained S emantics enhanced G raph C ontrastive L earning (FSGCL) in this paper. Concretely, FSGCL first introduces a motif-based graph construction, which employs graph motifs to extract diverse semantics existed in graphs from the perspective of input data. Then, the semantic-level contrastive task is explored to further enhance the utilization of fine-grained semantics from the perspective of model training. Experiments on five real-world datasets demonstrate the superiority of our proposed FSGCL over state-of-the-art methods. To make the results reproducible, we will make our codes public on GitHub after this paper is accepted.
{"title":"Fine-Grained Semantics Enhanced Contrastive Learning for Graphs","authors":"Youming Liu;Lin Shu;Chuan Chen;Zibin Zheng","doi":"10.1109/TKDE.2024.3466990","DOIUrl":"https://doi.org/10.1109/TKDE.2024.3466990","url":null,"abstract":"Graph contrastive learning defines a contrastive task to pull similar instances close and push dissimilar instances away. It learns discriminative node embeddings without supervised labels, which has aroused increasing attention in the past few years. Nevertheless, existing methods of graph contrastive learning ignore the differences between diverse semantics existed in graphs, which learn coarse-grained node embeddings and lead to sub-optimal performances on downstream tasks. To bridge this gap, we propose a novel \u0000<bold>F</b>\u0000ine-grained \u0000<bold>S</b>\u0000emantics enhanced \u0000<bold>G</b>\u0000raph \u0000<bold>C</b>\u0000ontrastive \u0000<bold>L</b>\u0000earning (FSGCL) in this paper. Concretely, FSGCL first introduces a motif-based graph construction, which employs graph motifs to extract diverse semantics existed in graphs from the perspective of input data. Then, the semantic-level contrastive task is explored to further enhance the utilization of fine-grained semantics from the perspective of model training. Experiments on five real-world datasets demonstrate the superiority of our proposed FSGCL over state-of-the-art methods. To make the results reproducible, we will make our codes public on GitHub after this paper is accepted.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"36 12","pages":"8238-8250"},"PeriodicalIF":8.9,"publicationDate":"2024-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600269","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-24DOI: 10.1109/TKDE.2024.3454328
Yi Zhang;Yuying Zhao;Zhaoqing Li;Xueqi Cheng;Yu Wang;Olivera Kotevska;Philip S. Yu;Tyler Derr
Graph Neural Networks (GNNs) have gained significant attention owing to their ability to handle graph-structured data and the improvement in practical applications. However, many of these models prioritize high utility performance, such as accuracy, with a lack of privacy consideration, which is a major concern in modern society where privacy attacks are rampant. To address this issue, researchers have started to develop privacy-preserving GNNs. Despite this progress, there is a lack of a comprehensive overview of the attacks and the techniques for preserving privacy in the graph domain. In this survey, we aim to address this gap by summarizing the attacks on graph data according to the targeted information, categorizing the privacy preservation techniques in GNNs, and reviewing the datasets and applications that could be used for analyzing/solving privacy issues in GNNs. We also outline potential directions for future research in order to build better privacy-preserving GNNs.
{"title":"A Survey on Privacy in Graph Neural Networks: Attacks, Preservation, and Applications","authors":"Yi Zhang;Yuying Zhao;Zhaoqing Li;Xueqi Cheng;Yu Wang;Olivera Kotevska;Philip S. Yu;Tyler Derr","doi":"10.1109/TKDE.2024.3454328","DOIUrl":"https://doi.org/10.1109/TKDE.2024.3454328","url":null,"abstract":"Graph Neural Networks (GNNs) have gained significant attention owing to their ability to handle graph-structured data and the improvement in practical applications. However, many of these models prioritize high utility performance, such as accuracy, with a lack of privacy consideration, which is a major concern in modern society where privacy attacks are rampant. To address this issue, researchers have started to develop privacy-preserving GNNs. Despite this progress, there is a lack of a comprehensive overview of the attacks and the techniques for preserving privacy in the graph domain. In this survey, we aim to address this gap by summarizing the attacks on graph data according to the targeted information, categorizing the privacy preservation techniques in GNNs, and reviewing the datasets and applications that could be used for analyzing/solving privacy issues in GNNs. We also outline potential directions for future research in order to build better privacy-preserving GNNs.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"36 12","pages":"7497-7515"},"PeriodicalIF":8.9,"publicationDate":"2024-09-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10693287","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142645552","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-23DOI: 10.1109/TKDE.2024.3466301
Hongyang Chen;Can Xu;Lingyu Zheng;Qiang Zhang;Xuemin Lin
Being the most cutting-edge generative methods, diffusion methods have shown great advances in wide generation tasks. Among them, graph generation attracts significant research attention for its broad application in real life. In our survey, we systematically and comprehensively review on diffusion-based graph generative methods. We first make a review on three mainstream paradigms of diffusion methods, which are denoising diffusion probabilistic models, score-based genrative models, and stochastic differential equations. Then we further categorize and introduce the latest applications of diffusion models on graphs. In the end, we point out some limitations of current studies and future directions of future explorations.
{"title":"Diffusion-Based Graph Generative Methods","authors":"Hongyang Chen;Can Xu;Lingyu Zheng;Qiang Zhang;Xuemin Lin","doi":"10.1109/TKDE.2024.3466301","DOIUrl":"https://doi.org/10.1109/TKDE.2024.3466301","url":null,"abstract":"Being the most cutting-edge generative methods, diffusion methods have shown great advances in wide generation tasks. Among them, graph generation attracts significant research attention for its broad application in real life. In our survey, we systematically and comprehensively review on diffusion-based graph generative methods. We first make a review on three mainstream paradigms of diffusion methods, which are denoising diffusion probabilistic models, score-based genrative models, and stochastic differential equations. Then we further categorize and introduce the latest applications of diffusion models on graphs. In the end, we point out some limitations of current studies and future directions of future explorations.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"36 12","pages":"7954-7972"},"PeriodicalIF":8.9,"publicationDate":"2024-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636363","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The robust multivariate time series anomaly detection can facilitate intelligent decisions and timely maintenance in various kinds of monitor systems. However, the robustness is highly restricted by the stochasticity in multivariate time series, which is summarized as temporal stochasticity and spatial stochasticity specifically. In this paper, we explicitly model the temporal stochasticity variables and the latent graph relationship variables into a unified graphical framework, which can achieve better robustness to dynamicity from both the spatial and temporal perspective. First, within the spatial encoder, every connection exists or not is modeled as a binary stochastic variable, and the graph structure can be learnt automatically. Then, the temporal encoder would embed the highly structured time series into latent stochastic variables to capture both complex temporal dependencies and neighbors information. Moreover, we design a history-future combined anomaly score mechanism with both reconstruction decoder and forecasting decoder to improve the anomaly detection performance. By weighting the historical anomaly factor, the future anomaly factor, and the prediction error of current timestamp, the anomaly detection at current timestamp could be more sensitive to anomaly detection. Finally, extensive experiments on three publicly available anomaly detection datasets demonstrate our proposed method can achieve the best performance in terms of recall and F1 compared with state-of-the-arts baselines.
稳健的多变量时间序列异常检测有助于各类监控系统的智能决策和及时维护。然而，多变量时间序列中的随机性极大地限制了其鲁棒性，具体可概括为时间随机性和空间随机性。本文将时间随机性变量和潜在图关系变量明确建模到一个统一的图框架中，从而从空间和时间两个角度实现更好的动态鲁棒性。首先，在空间编码器中，每个连接存在与否都被建模为二元随机变量，图结构可以自动学习。然后，时间编码器将高度结构化的时间序列嵌入到潜在随机变量中，以捕捉复杂的时间依赖性和邻近信息。此外，我们还设计了一种历史与未来相结合的异常评分机制，同时使用重构解码器和预测解码器来提高异常检测性能。通过对历史异常因子、未来异常因子和当前时间戳的预测误差进行加权，可以提高当前时间戳的异常检测灵敏度。最后，在三个公开的异常检测数据集上进行的大量实验表明，与同行相比，我们提出的方法在召回率和 F1 方面都能达到最佳性能。
{"title":"Rethinking Robust Multivariate Time Series Anomaly Detection: A Hierarchical Spatio-Temporal Variational Perspective","authors":"Xiao Zhang;Shuqing Xu;Huashan Chen;Zekai Chen;Fuzhen Zhuang;Hui Xiong;Dongxiao Yu","doi":"10.1109/TKDE.2024.3466291","DOIUrl":"https://doi.org/10.1109/TKDE.2024.3466291","url":null,"abstract":"The robust multivariate time series anomaly detection can facilitate intelligent decisions and timely maintenance in various kinds of monitor systems. However, the robustness is highly restricted by the stochasticity in multivariate time series, which is summarized as \u0000<italic>temporal stochasticity</i>\u0000 and \u0000<italic>spatial stochasticity</i>\u0000 specifically. In this paper, we explicitly model the temporal stochasticity variables and the latent graph relationship variables into a unified graphical framework, which can achieve better robustness to dynamicity from both the spatial and temporal perspective. First, within the spatial encoder, every connection exists or not is modeled as a binary stochastic variable, and the graph structure can be learnt automatically. Then, the temporal encoder would embed the highly structured time series into latent stochastic variables to capture both complex temporal dependencies and neighbors information. Moreover, we design a history-future combined anomaly score mechanism with both reconstruction decoder and forecasting decoder to improve the anomaly detection performance. By weighting the historical anomaly factor, the future anomaly factor, and the prediction error of current timestamp, the anomaly detection at current timestamp could be more sensitive to anomaly detection. Finally, extensive experiments on three publicly available anomaly detection datasets demonstrate our proposed method can achieve the best performance in terms of recall and F1 compared with state-of-the-arts baselines.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"36 12","pages":"9136-9149"},"PeriodicalIF":8.9,"publicationDate":"2024-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142600145","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-23DOI: 10.1109/TKDE.2024.3466508
Xuexiong Luo;Sheng Zhang;Jia Wu;Hongyang Chen;Hao Peng;Chuan Zhou;Zhao Li;Shan Xue;Jian Yang
Graph pooling technique as the essential component of graph neural networks has gotten increasing attention recently and it aims to learn graph-level representations for the whole graph. Besides, graph pooling is important in graph classification and graph generation tasks. However, current graph pooling methods mainly coarsen a sequence of small-sized graphs to capture hierarchical structures, potentially resulting in the deterioration of the global structure of the original graph and influencing the quality of graph representations. Furthermore, these methods artificially select the number of graph pooling layers for different graph datasets rather than considering each graph individually. In reality, the structure and size differences among graphs necessitate a specific number of graph pooling layers for each graph. In this work, we propose reinforced pooling graph neural networks via adaptive hybrid graph coarsening networks. Specifically, we design a hybrid graph coarsening strategy to coarsen redundant structures of the original graph while retaining the global structure. In addition, we introduce multi-agent reinforcement learning to adaptively perform the graph coarsening process to extract the most representative coarsened graph for each graph, enhancing the quality of graph-level representations. Finally, we design graph-level contrast to improve the preservation of global information in graph-level representations. Extensive experiments with rich baselines on six benchmark datasets show the effectiveness of ReiPool 1 .
{"title":"ReiPool: Reinforced Pooling Graph Neural Networks for Graph-Level Representation Learning","authors":"Xuexiong Luo;Sheng Zhang;Jia Wu;Hongyang Chen;Hao Peng;Chuan Zhou;Zhao Li;Shan Xue;Jian Yang","doi":"10.1109/TKDE.2024.3466508","DOIUrl":"https://doi.org/10.1109/TKDE.2024.3466508","url":null,"abstract":"Graph pooling technique as the essential component of graph neural networks has gotten increasing attention recently and it aims to learn graph-level representations for the whole graph. Besides, graph pooling is important in graph classification and graph generation tasks. However, current graph pooling methods mainly coarsen a sequence of small-sized graphs to capture hierarchical structures, potentially resulting in the deterioration of the global structure of the original graph and influencing the quality of graph representations. Furthermore, these methods artificially select the number of graph pooling layers for different graph datasets rather than considering each graph individually. In reality, the structure and size differences among graphs necessitate a specific number of graph pooling layers for each graph. In this work, we propose reinforced pooling graph neural networks via adaptive hybrid graph coarsening networks. Specifically, we design a hybrid graph coarsening strategy to coarsen redundant structures of the original graph while retaining the global structure. In addition, we introduce multi-agent reinforcement learning to adaptively perform the graph coarsening process to extract the most representative coarsened graph for each graph, enhancing the quality of graph-level representations. Finally, we design graph-level contrast to improve the preservation of global information in graph-level representations. Extensive experiments with rich baselines on six benchmark datasets show the effectiveness of ReiPool\u0000<sup>1</sup>\u0000.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"36 12","pages":"9109-9122"},"PeriodicalIF":8.9,"publicationDate":"2024-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636281","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：