Pub Date : 2024-10-29DOI: 10.1109/TKDE.2024.3487523
Yi Yang;John P. Lalor;Ahmed Abbasi;Daniel Dajun Zeng
Topic modeling is a commonly used text analysis tool for discovering latent topics in a text corpus. However, while topics in a text corpus often exhibit a hierarchical structure (e.g., cellphone is a sub-topic of electronics), most topic modeling methods assume a flat topic structure that ignores the hierarchical dependency among topics, or utilize a predefined topic hierarchy. In this work, we present a novel Hierarchical Deep Document Model (HDDM) to learn topic hierarchies using a variational autoencoder framework. We propose a novel objective function, sum of log likelihood, instead of the widely used evidence lower bound, to facilitate the learning of hierarchical latent topic structure. The proposed objective function can directly model and optimize the hierarchical topic-word distributions at all topic levels. We conduct experiments on four real-world text datasets to evaluate the topic modeling capability of the proposed HDDM method compared to state-of-the-art hierarchical topic modeling benchmarks. Experimental results show that HDDM achieves considerable improvement over benchmarks and is capable of learning meaningful topics and topic hierarchies. To further demonstrate the practical utility of HDDM, we apply it to a real-world medical notes dataset for clinical prediction. Experimental results show that HDDM can better summarize topics in medical notes, resulting in more accurate clinical predictions.
{"title":"Hierarchical Deep Document Model","authors":"Yi Yang;John P. Lalor;Ahmed Abbasi;Daniel Dajun Zeng","doi":"10.1109/TKDE.2024.3487523","DOIUrl":"https://doi.org/10.1109/TKDE.2024.3487523","url":null,"abstract":"Topic modeling is a commonly used text analysis tool for discovering latent topics in a text corpus. However, while topics in a text corpus often exhibit a hierarchical structure (e.g., cellphone is a sub-topic of electronics), most topic modeling methods assume a flat topic structure that ignores the hierarchical dependency among topics, or utilize a predefined topic hierarchy. In this work, we present a novel Hierarchical Deep Document Model (HDDM) to learn topic hierarchies using a variational autoencoder framework. We propose a novel objective function, sum of log likelihood, instead of the widely used evidence lower bound, to facilitate the learning of hierarchical latent topic structure. The proposed objective function can directly model and optimize the hierarchical topic-word distributions at all topic levels. We conduct experiments on four real-world text datasets to evaluate the topic modeling capability of the proposed HDDM method compared to state-of-the-art hierarchical topic modeling benchmarks. Experimental results show that HDDM achieves considerable improvement over benchmarks and is capable of learning meaningful topics and topic hierarchies. To further demonstrate the practical utility of HDDM, we apply it to a real-world medical notes dataset for clinical prediction. Experimental results show that HDDM can better summarize topics in medical notes, resulting in more accurate clinical predictions.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"37 1","pages":"351-364"},"PeriodicalIF":8.9,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142810295","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Modern recommender systems derive predictions from an interaction graph that links users and items. To this end, many of today's state-of-the-art systems use graph neural networks (GNNs) to learn effective representations of these graphs under the assumption of homophily, i.e., the idea that similar users will sit close to each other in the graph. However, recent studies have revealed that real-world recommendation graphs are often heterophilous, i.e., dissimilar users will also often sit close to each other. One of the reasons for this heterophilia is shilling attacks that obscure the inherent characteristics of the graph and make the derived recommendations less accurate as a consequence. Hence, to cope with low homophily in recommender systems, we propose a recommendation model called PGT4Rec that is based on a Partitioned Graph Transformer. The model integrates label information into the learning process, which allows discriminative neighbourhoods of users to be generated. As such, the framework can both detect shilling attacks and predict user ratings for items. Extensive experiments on real and synthetic datasets show PGT4Rec as not only providing superior performance in these two tasks but also significant robustness to a range of adversarial conditions.
{"title":"Handling Low Homophily in Recommender Systems With Partitioned Graph Transformer","authors":"Thanh Tam Nguyen;Thanh Toan Nguyen;Matthias Weidlich;Jun Jo;Quoc Viet Hung Nguyen;Hongzhi Yin;Alan Wee-Chung Liew","doi":"10.1109/TKDE.2024.3485880","DOIUrl":"https://doi.org/10.1109/TKDE.2024.3485880","url":null,"abstract":"Modern recommender systems derive predictions from an interaction graph that links users and items. To this end, many of today's state-of-the-art systems use graph neural networks (GNNs) to learn effective representations of these graphs under the assumption of homophily, i.e., the idea that similar users will sit close to each other in the graph. However, recent studies have revealed that real-world recommendation graphs are often heterophilous, i.e., dissimilar users will also often sit close to each other. One of the reasons for this heterophilia is shilling attacks that obscure the inherent characteristics of the graph and make the derived recommendations less accurate as a consequence. Hence, to cope with low homophily in recommender systems, we propose a recommendation model called PGT4Rec that is based on a Partitioned Graph Transformer. The model integrates label information into the learning process, which allows discriminative neighbourhoods of users to be generated. As such, the framework can both detect shilling attacks and predict user ratings for items. Extensive experiments on real and synthetic datasets show PGT4Rec as not only providing superior performance in these two tasks but also significant robustness to a range of adversarial conditions.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"37 1","pages":"334-350"},"PeriodicalIF":8.9,"publicationDate":"2024-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142797954","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Existing geometric knowledge graph embedding methods employ various relational transformations, such as translation, rotation, and projection, to model different relation patterns, which aims to enhance the expressiveness of models. In contrast to current approaches that treat the expressiveness of the model as a binary issue, we aim to delve deeper into analyzing the level of difficulty in which geometric knowledge graph embedding models can represent relation patterns. In this paper, we provide a theoretical analysis framework that measures the expressiveness of the model in relation patterns by quantifying the size of the solution space of linear equation systems. Additionally, we propose a mechanism for imposing relational constraints on geometric knowledge graph embedding models by setting “traps” near relational optimal solutions, which enables the model to better converge to the optimal solution. Empirically, we analyze and compare several typical knowledge graph embedding models with different geometric algebras, revealing that some models have insufficient solution space due to their design, which leads to performance weaknesses. We also demonstrate that the proposed relational constraint operations can improve the performance of certain relation patterns. The experimental results on public benchmarks and relation pattern specified dataset are consistent with our theoretical analysis.
{"title":"Expressiveness Analysis and Enhancing Framework for Geometric Knowledge Graph Embedding Models","authors":"Tengwei Song;Long Yin;Yang Liu;Long Liao;Jie Luo;Zhiqiang Xu","doi":"10.1109/TKDE.2024.3486915","DOIUrl":"https://doi.org/10.1109/TKDE.2024.3486915","url":null,"abstract":"Existing geometric knowledge graph embedding methods employ various relational transformations, such as translation, rotation, and projection, to model different relation patterns, which aims to enhance the expressiveness of models. In contrast to current approaches that treat the expressiveness of the model as a binary issue, we aim to delve deeper into analyzing the level of difficulty in which geometric knowledge graph embedding models can represent relation patterns. In this paper, we provide a theoretical analysis framework that measures the expressiveness of the model in relation patterns by quantifying the size of the solution space of linear equation systems. Additionally, we propose a mechanism for imposing relational constraints on geometric knowledge graph embedding models by setting “traps” near relational optimal solutions, which enables the model to better converge to the optimal solution. Empirically, we analyze and compare several typical knowledge graph embedding models with different geometric algebras, revealing that some models have insufficient solution space due to their design, which leads to performance weaknesses. We also demonstrate that the proposed relational constraint operations can improve the performance of certain relation patterns. The experimental results on public benchmarks and relation pattern specified dataset are consistent with our theoretical analysis.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"37 1","pages":"306-318"},"PeriodicalIF":8.9,"publicationDate":"2024-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142797991","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-25DOI: 10.1109/TKDE.2024.3485107
Li Yuan;Yi Cai;Jingyu Xu;Qing Li;Tao Wang
Joint multimodal entity-relation extraction (JMERE) is a challenging task that involves two joint subtasks, i.e., named entity recognition and relation extraction, from multimodal data such as text sentences with associated images. Previous JMERE methods have primarily employed 1) pipeline models, which apply pre-trained unimodal models separately and ignore the interaction between tasks, or 2) word-pair relation tagging methods, which neglect neighboring word pairs. To address these limitations, we propose a fine-grained network for JMERE. Specifically, we introduce a fine-grained alignment module that utilizes a phrase-patch to establish connections between text phrases and visual objects. This module can learn consistent multimodal representations from multimodal data. Furthermore, we address the task-irrelevant image information issue by proposing a gate fusion module, which mitigates the impact of image noise and ensures a balanced representation between image objects and text representations. Furthermore, we design a multi-word decoder that enables ensemble prediction of tags for each word pair. This approach leverages the predicted results of neighboring word pairs, improving the ability to extract multi-word entities. Evaluation results from a series of experiments demonstrate the superiority of our proposed model over state-of-the-art models in JMERE.
{"title":"A Fine-Grained Network for Joint Multimodal Entity-Relation Extraction","authors":"Li Yuan;Yi Cai;Jingyu Xu;Qing Li;Tao Wang","doi":"10.1109/TKDE.2024.3485107","DOIUrl":"https://doi.org/10.1109/TKDE.2024.3485107","url":null,"abstract":"Joint multimodal entity-relation extraction (JMERE) is a challenging task that involves two joint subtasks, i.e., named entity recognition and relation extraction, from multimodal data such as text sentences with associated images. Previous JMERE methods have primarily employed 1) pipeline models, which apply pre-trained unimodal models separately and ignore the interaction between tasks, or 2) word-pair relation tagging methods, which neglect neighboring word pairs. To address these limitations, we propose a fine-grained network for JMERE. Specifically, we introduce a fine-grained alignment module that utilizes a phrase-patch to establish connections between text phrases and visual objects. This module can learn consistent multimodal representations from multimodal data. Furthermore, we address the task-irrelevant image information issue by proposing a gate fusion module, which mitigates the impact of image noise and ensures a balanced representation between image objects and text representations. Furthermore, we design a multi-word decoder that enables ensemble prediction of tags for each word pair. This approach leverages the predicted results of neighboring word pairs, improving the ability to extract multi-word entities. Evaluation results from a series of experiments demonstrate the superiority of our proposed model over state-of-the-art models in JMERE.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"37 1","pages":"1-14"},"PeriodicalIF":8.9,"publicationDate":"2024-10-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142797912","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-25DOI: 10.1109/TKDE.2024.3486530
Guangjie Zeng;Hao Peng;Angsheng Li;Jia Wu;Chunyang Liu;Philip S. Yu
Semi-supervised clustering leverages prior information in the form of constraints to achieve higher-quality clustering outcomes. However, most existing methods struggle with large-scale datasets owing to their high time and space complexity. Moreover, they encounter the challenge of seamlessly integrating various constraints, thereby limiting their applicability. In this paper, we present S calable S emi-supervised clustering via S tructural E ntropy (SSSE), a novel method that tackles scalable datasets with different types of constraints from diverse sources to perform both semi-supervised partitioning and hierarchical clustering, which is fully explainable compared to deep learning-based methods. Specifically, we design objectives based on structural entropy, integrating constraints for semi-supervised partitioning and hierarchical clustering. To achieve scalability on data size, we develop efficient algorithms based on graph sampling to reduce the time and space complexity. To achieve generalization on constraint types, we formulate a uniform view for widely used pairwise and label constraints. Extensive experiments on real-world clustering datasets at different scales demonstrate the superiority of SSSE in clustering accuracy and scalability with different constraints. Additionally, Cell clustering experiments on single-cell RNA-seq datasets demonstrate the functionality of SSSE for biological data analysis.
{"title":"Scalable Semi-Supervised Clustering via Structural Entropy With Different Constraints","authors":"Guangjie Zeng;Hao Peng;Angsheng Li;Jia Wu;Chunyang Liu;Philip S. Yu","doi":"10.1109/TKDE.2024.3486530","DOIUrl":"https://doi.org/10.1109/TKDE.2024.3486530","url":null,"abstract":"Semi-supervised clustering leverages prior information in the form of constraints to achieve higher-quality clustering outcomes. However, most existing methods struggle with large-scale datasets owing to their high time and space complexity. Moreover, they encounter the challenge of seamlessly integrating various constraints, thereby limiting their applicability. In this paper, we present \u0000<underline>S</u>\u0000calable \u0000<underline>S</u>\u0000emi-supervised clustering via \u0000<underline>S</u>\u0000tructural \u0000<underline>E</u>\u0000ntropy (SSSE), a novel method that tackles scalable datasets with different types of constraints from diverse sources to perform both semi-supervised partitioning and hierarchical clustering, which is fully explainable compared to deep learning-based methods. Specifically, we design objectives based on structural entropy, integrating constraints for semi-supervised partitioning and hierarchical clustering. To achieve scalability on data size, we develop efficient algorithms based on graph sampling to reduce the time and space complexity. To achieve generalization on constraint types, we formulate a uniform view for widely used pairwise and label constraints. Extensive experiments on real-world clustering datasets at different scales demonstrate the superiority of SSSE in clustering accuracy and scalability with different constraints. Additionally, Cell clustering experiments on single-cell RNA-seq datasets demonstrate the functionality of SSSE for biological data analysis.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"37 1","pages":"478-492"},"PeriodicalIF":8.9,"publicationDate":"2024-10-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142810280","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Graph clustering technique is highly effective in detecting complex-shaped clusters, in which graph building is a crucial step. Nevertheless, building a reasonable graph that can exhibit high connectivity within clusters and low connectivity across clusters is challenging. Herein, we design a max-ascent-angle graph called the “Y-graph”, a high-sparse graph that automatically allocates dense edges within clusters and sparse edges across clusters, regardless of their shapes or dimensionality. In the graph, every point $x$ is allowed to connect its nearest higher-density neighbor $delta$ , and another higher-density neighbor $gamma$ , satisfying that the angle $angle delta xgamma$ is the largest, called “max-ascent-angle”. By seeking the max-ascent-angle, points are automatically connected as the Y-graph, which is a reasonable graph that can effectively balance inter-cluster connectivity and intra-cluster non-connectivity. Besides, an edge weight function is designed to capture the similarity of the neighbor probability distribution, which effectively represents the density connectivity between points. By employing the Normalized-Cut (Ncut) technique, a Ncut-Y algorithm is proposed. Benefiting from the excellent performance of Y-graph, Ncut-Y can fast seek and cut the edges located in the low-density boundaries between clusters, thereby, capturing clusters effectively. Experimental results on both synthetic and real datasets demonstrate the effectiveness of Y-graph and Ncut-Y.
{"title":"Y-Graph: A Max-Ascent-Angle Graph for Detecting Clusters","authors":"Junyi Guan;Sheng Li;Xiongxiong He;Jiajia Chen;Yangyang Zhao;Yuxuan Zhang","doi":"10.1109/TKDE.2024.3486221","DOIUrl":"https://doi.org/10.1109/TKDE.2024.3486221","url":null,"abstract":"Graph clustering technique is highly effective in detecting complex-shaped clusters, in which graph building is a crucial step. Nevertheless, building a reasonable graph that can exhibit high connectivity within clusters and low connectivity across clusters is challenging. Herein, we design a max-ascent-angle graph called the “Y-graph”, a high-sparse graph that automatically allocates dense edges within clusters and sparse edges across clusters, regardless of their shapes or dimensionality. In the graph, every point \u0000<inline-formula><tex-math>$x$</tex-math></inline-formula>\u0000 is allowed to connect its nearest higher-density neighbor \u0000<inline-formula><tex-math>$delta$</tex-math></inline-formula>\u0000, and another higher-density neighbor \u0000<inline-formula><tex-math>$gamma$</tex-math></inline-formula>\u0000, satisfying that the angle \u0000<inline-formula><tex-math>$angle delta xgamma$</tex-math></inline-formula>\u0000 is the largest, called “max-ascent-angle”. By seeking the max-ascent-angle, points are automatically connected as the Y-graph, which is a reasonable graph that can effectively balance inter-cluster connectivity and intra-cluster non-connectivity. Besides, an edge weight function is designed to capture the similarity of the neighbor probability distribution, which effectively represents the density connectivity between points. By employing the Normalized-Cut (Ncut) technique, a Ncut-Y algorithm is proposed. Benefiting from the excellent performance of Y-graph, Ncut-Y can fast seek and cut the edges located in the low-density boundaries between clusters, thereby, capturing clusters effectively. Experimental results on both synthetic and real datasets demonstrate the effectiveness of Y-graph and Ncut-Y.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"37 1","pages":"542-556"},"PeriodicalIF":8.9,"publicationDate":"2024-10-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142810359","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
When handling streaming graphs, existing graph representation learning models encounter a catastrophic forgetting problem, where previously learned knowledge of these models is easily overwritten when learning with newly incoming graphs. In response, Continual Graph Learning (CGL) emerges as a novel paradigm enabling graph representation learning from static to streaming graphs. Our prior work, Condense and Train (CaT) (Liu et al. 2023) is a replay-based CGL framework with a balanced continual learning procedure, which designs a small yet effective memory bank for replaying data by condensing incoming graphs. Although the CaT alleviates the catastrophic forgetting problem, there exist three issues: (1) The graph condensation algorithm derived in CaT only focuses on labelled nodes while neglecting abundant information carried by unlabelled nodes; (2) The continual training scheme of the CaT overemphasises on the previously learned knowledge, limiting the model capacity to learn from newly added memories; (3) Both the condensation process and replaying process of the CaT are time-consuming. In this paper, we propose a P s U do-label guided M emory b A nk (PUMA) CGL framework, extending from the CaT to enhance its efficiency and effectiveness by overcoming the above-mentioned weaknesses and limits. To fully exploit the information in a graph, PUMA expands the coverage of nodes during graph condensation with both labelled and unlabelled nodes. Furthermore, a training-from-scratch strategy is proposed to upgrade the previous continual learning scheme for a balanced training between the historical and the new graphs. Besides, PUMA uses a one-time prorogation and wide graph encoders to accelerate the graph condensation and the graph encoding process in the training stage to improve the efficiency of the whole framework. Extensive experiments on seven datasets for the node classification task demonstrate the state-of-the-art performance and efficiency over existing methods.
在处理流图时，现有的图表示学习模型会遇到灾难性的遗忘问题，在使用新传入的图学习时，以前学习过的这些模型的知识很容易被覆盖。作为回应，持续图学习（CGL）作为一种新的范式出现，使图表示能够从静态图学习到流图。我们之前的工作，冷凝和训练（CaT） （Liu et al. 2023）是一个基于重播的CGL框架，具有平衡的持续学习过程，它设计了一个小而有效的记忆库，通过压缩传入的图来重播数据。虽然CaT缓解了灾难性遗忘问题，但存在三个问题：(1)CaT导出的图凝聚算法只关注有标记的节点，而忽略了未标记节点携带的丰富信息；(2) CaT的持续训练方案过分强调之前学习过的知识，限制了模型从新增记忆中学习的能力；(3) CaT的冷凝过程和重放过程都是耗时的。在本文中，我们提出了一个PsUdo-label引导记忆库（PUMA） CGL框架，该框架从CaT扩展而来，通过克服上述弱点和局限性来提高其效率和有效性。为了充分利用图中的信息，PUMA在图凝聚过程中扩展了节点的覆盖范围，包括标记节点和未标记节点。此外，提出了一种从头开始的训练策略，以升级以前的连续学习方案，实现历史图和新图之间的平衡训练。PUMA在训练阶段使用了一次延图和宽图编码器，加速了图的凝聚和图的编码过程，提高了整个框架的效率。在7个数据集上对节点分类任务进行了广泛的实验，证明了该方法优于现有方法的最先进性能和效率。
{"title":"PUMA: Efficient Continual Graph Learning for Node Classification With Graph Condensation","authors":"Yilun Liu;Ruihong Qiu;Yanran Tang;Hongzhi Yin;Zi Huang","doi":"10.1109/TKDE.2024.3485691","DOIUrl":"https://doi.org/10.1109/TKDE.2024.3485691","url":null,"abstract":"When handling streaming graphs, existing graph representation learning models encounter a catastrophic forgetting problem, where previously learned knowledge of these models is easily overwritten when learning with newly incoming graphs. In response, Continual Graph Learning (CGL) emerges as a novel paradigm enabling graph representation learning from static to streaming graphs. Our prior work, Condense and Train (CaT) (Liu et al. 2023) is a replay-based CGL framework with a balanced continual learning procedure, which designs a small yet effective memory bank for replaying data by condensing incoming graphs. Although the CaT alleviates the catastrophic forgetting problem, there exist three issues: (1) The graph condensation algorithm derived in CaT only focuses on labelled nodes while neglecting abundant information carried by unlabelled nodes; (2) The continual training scheme of the CaT overemphasises on the previously learned knowledge, limiting the model capacity to learn from newly added memories; (3) Both the condensation process and replaying process of the CaT are time-consuming. In this paper, we propose a \u0000<bold>P</b>\u0000s\u0000<bold>U</b>\u0000do-label guided \u0000<bold>M</b>\u0000emory b\u0000<bold>A</b>\u0000nk (PUMA) CGL framework, extending from the CaT to enhance its efficiency and effectiveness by overcoming the above-mentioned weaknesses and limits. To fully exploit the information in a graph, PUMA expands the coverage of nodes during graph condensation with both labelled and unlabelled nodes. Furthermore, a training-from-scratch strategy is proposed to upgrade the previous continual learning scheme for a balanced training between the historical and the new graphs. Besides, PUMA uses a one-time prorogation and wide graph encoders to accelerate the graph condensation and the graph encoding process in the training stage to improve the efficiency of the whole framework. Extensive experiments on seven datasets for the node classification task demonstrate the state-of-the-art performance and efficiency over existing methods.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"37 1","pages":"449-461"},"PeriodicalIF":8.9,"publicationDate":"2024-10-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142810363","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Domain generalization (DG) tasks aim to learn cross-domain models from source domains and apply them to unknown target domains. Recent research has demonstrated that diverse and rich source domain samples can enhance domain generalization capability. This work argues that the impact of each sample on the model's generalization ability varies. Even a small-scale but high-quality dataset can achieve a notable level of generalization. Motivated by this, we propose a domain-adversarial active learning (DAAL) algorithm for classification tasks in DG. First, we analyze that the objective of DG tasks is to maximize the inter-class distance within the same domain and minimize the intra-class distance across different domains. We design a domain adversarial selection method that prioritizes challenging samples in an active learning (AL) framework. Second, we hypothesize that even in a converged model, some feature subsets lack discriminatory power within each domain. We develop a method to identify and optimize these feature subsets, thereby maximizing inter-class distance of features. Lastly, We experimentally compare our DAAL algorithm with various DG and AL algorithms across four datasets. The results demonstrate that the DAAL algorithm can achieve strong generalization ability with fewer data resources, thereby significantly reducing data annotation costs in DG tasks.
{"title":"Domain Adversarial Active Learning for Domain Generalization Classification","authors":"Jianting Chen;Ling Ding;Yunxiao Yang;Zaiyuan Di;Yang Xiang","doi":"10.1109/TKDE.2024.3486204","DOIUrl":"https://doi.org/10.1109/TKDE.2024.3486204","url":null,"abstract":"Domain generalization (DG) tasks aim to learn cross-domain models from source domains and apply them to unknown target domains. Recent research has demonstrated that diverse and rich source domain samples can enhance domain generalization capability. This work argues that the impact of each sample on the model's generalization ability varies. Even a small-scale but high-quality dataset can achieve a notable level of generalization. Motivated by this, we propose a domain-adversarial active learning (DAAL) algorithm for classification tasks in DG. First, we analyze that the objective of DG tasks is to maximize the inter-class distance within the same domain and minimize the intra-class distance across different domains. We design a domain adversarial selection method that prioritizes challenging samples in an active learning (AL) framework. Second, we hypothesize that even in a converged model, some feature subsets lack discriminatory power within each domain. We develop a method to identify and optimize these feature subsets, thereby maximizing inter-class distance of features. Lastly, We experimentally compare our DAAL algorithm with various DG and AL algorithms across four datasets. The results demonstrate that the DAAL algorithm can achieve strong generalization ability with fewer data resources, thereby significantly reducing data annotation costs in DG tasks.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"37 1","pages":"226-238"},"PeriodicalIF":8.9,"publicationDate":"2024-10-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142798045","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-24DOI: 10.1109/TKDE.2024.3485736
Dengsheng Wu;Huidong Wu;Jianping Li
Graph Neural Networks (GNNs) have gained attention for their ability in capturing node interactions to generate node representations. However, their performances are frequently restricted in real-world directed networks with natural hierarchical structures. Most current GNNs incorporate information from immediate neighbors or within predefined receptive fields, potentially overlooking long-range dependencies inherent in hierarchical structures. They also tend to neglect node adaptability, which varies based on their positions. To address these limitations, we propose a new model called Hierarchy-Aware Adaptive Graph Neural Network (HAGNN) to adaptively capture hierarchical long-range dependencies. Technically, HAGNN creates a hierarchical structure based on directional pair-wise node interactions, revealing underlying hierarchical relationships among nodes. The inferred hierarchy helps to identify certain key nodes, named Source Hubs in our research, which serve as hierarchical contexts for individual nodes. Shortcuts adaptively connect these Source Hubs with distant nodes, enabling efficient message passing for informative long-range interactions. Through comprehensive experiments across multiple datasets, our proposed model outperforms several baseline methods, thus establishing a new state-of-the-art in performance. Further analysis demonstrates the effectiveness of our approach in capturing relevant adaptive hierarchical contexts, leading to improved and explainable node representation.
{"title":"Hierarchy-Aware Adaptive Graph Neural Network","authors":"Dengsheng Wu;Huidong Wu;Jianping Li","doi":"10.1109/TKDE.2024.3485736","DOIUrl":"https://doi.org/10.1109/TKDE.2024.3485736","url":null,"abstract":"Graph Neural Networks (GNNs) have gained attention for their ability in capturing node interactions to generate node representations. However, their performances are frequently restricted in real-world directed networks with natural hierarchical structures. Most current GNNs incorporate information from immediate neighbors or within predefined receptive fields, potentially overlooking long-range dependencies inherent in hierarchical structures. They also tend to neglect node adaptability, which varies based on their positions. To address these limitations, we propose a new model called Hierarchy-Aware Adaptive Graph Neural Network (HAGNN) to adaptively capture hierarchical long-range dependencies. Technically, HAGNN creates a hierarchical structure based on directional pair-wise node interactions, revealing underlying hierarchical relationships among nodes. The inferred hierarchy helps to identify certain key nodes, named Source Hubs in our research, which serve as hierarchical contexts for individual nodes. Shortcuts adaptively connect these Source Hubs with distant nodes, enabling efficient message passing for informative long-range interactions. Through comprehensive experiments across multiple datasets, our proposed model outperforms several baseline methods, thus establishing a new state-of-the-art in performance. Further analysis demonstrates the effectiveness of our approach in capturing relevant adaptive hierarchical contexts, leading to improved and explainable node representation.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"37 1","pages":"365-378"},"PeriodicalIF":8.9,"publicationDate":"2024-10-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142810278","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-24DOI: 10.1109/TKDE.2024.3486310
Tongfeng Weng;Yumeng Liu;Mo Sha;Xinyuan Chen;Xu Zhou;Kenli Li;Kian-Lee Tan
This paper addresses the pressing need for effective k-tips decomposition in dynamic bipartite graphs, a crucial aspect of real-time applications that analyze and mine binary relationship patterns. Recognizing the dynamic nature of these graphs, our study is the first to provide a solution for k-tips decomposition in such evolving environments. We introduce a pioneering projection-based algorithm, coupled with advanced incremental maintenance strategies for edge modifications, tailored specifically for dynamic graphs. This novel approach not only fills a significant gap in the analysis of dynamic bipartite graphs but also substantially enhances the accuracy and timeliness of data-driven decisions in critical areas like public health. Our contributions set a new benchmark in the field, paving the way for more nuanced and responsive analyses in various domains reliant on dynamic data interpretation.
{"title":"Efficient Projection-Based Algorithms for Tip Decomposition on Dynamic Bipartite Graphs","authors":"Tongfeng Weng;Yumeng Liu;Mo Sha;Xinyuan Chen;Xu Zhou;Kenli Li;Kian-Lee Tan","doi":"10.1109/TKDE.2024.3486310","DOIUrl":"https://doi.org/10.1109/TKDE.2024.3486310","url":null,"abstract":"This paper addresses the pressing need for effective k-tips decomposition in dynamic bipartite graphs, a crucial aspect of real-time applications that analyze and mine binary relationship patterns. Recognizing the dynamic nature of these graphs, our study is the first to provide a solution for k-tips decomposition in such evolving environments. We introduce a pioneering projection-based algorithm, coupled with advanced incremental maintenance strategies for edge modifications, tailored specifically for dynamic graphs. This novel approach not only fills a significant gap in the analysis of dynamic bipartite graphs but also substantially enhances the accuracy and timeliness of data-driven decisions in critical areas like public health. Our contributions set a new benchmark in the field, paving the way for more nuanced and responsive analyses in various domains reliant on dynamic data interpretation.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"37 2","pages":"626-640"},"PeriodicalIF":8.9,"publicationDate":"2024-10-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142940961","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：