Zhifei Li , Yue Jian , Zengcan Xue , Yumin Zheng , Miao Zhang , Yan Zhang , Xiaoju Hou , Xiaoguang Wang
{"title":"具有局部结构的文本增强型知识图谱表示学习","authors":"Zhifei Li , Yue Jian , Zengcan Xue , Yumin Zheng , Miao Zhang , Yan Zhang , Xiaoju Hou , Xiaoguang Wang","doi":"10.1016/j.ipm.2024.103797","DOIUrl":null,"url":null,"abstract":"<div><p>Knowledge graph representation learning entails transforming entities and relationships within a knowledge graph into vectors to enhance downstream tasks. The rise of pre-trained language models has recently promoted text-based approaches for knowledge graph representation learning. However, these methods often need more structural information on knowledge graphs, prompting the challenge of integrating graph structure knowledge into text-based methodologies. To tackle this issue, we introduce a text-enhanced model with local structure (TEGS) that embeds local graph structure details from the knowledge graph into the text encoder. TEGS integrates <em>k</em>-hop neighbor entity information into the text encoder and employs a decoupled attention mechanism to blend relative position encoding and text semantics. This strategy augments learnable content through graph structure information and mitigates the impact of semantic ambiguity via the decoupled attention mechanism. Experimental findings demonstrate TEGS’s effectiveness at fusing graph structure information, resulting in state-of-the-art performance across three datasets in link prediction tasks. In terms of Hit@1, when compared to the previous text-based models, our model demonstrated improvements of 2.1% on WN18RR, 2.4% on FB15k-237, and 2.7% on the NELL-One dataset. Our code is made publicly available on <span>https://github.com/HubuKG/TEGS</span><svg><path></path></svg>.</p></div>","PeriodicalId":50365,"journal":{"name":"Information Processing & Management","volume":null,"pages":null},"PeriodicalIF":7.4000,"publicationDate":"2024-06-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Text-enhanced knowledge graph representation learning with local structure\",\"authors\":\"Zhifei Li , Yue Jian , Zengcan Xue , Yumin Zheng , Miao Zhang , Yan Zhang , Xiaoju Hou , Xiaoguang Wang\",\"doi\":\"10.1016/j.ipm.2024.103797\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><p>Knowledge graph representation learning entails transforming entities and relationships within a knowledge graph into vectors to enhance downstream tasks. The rise of pre-trained language models has recently promoted text-based approaches for knowledge graph representation learning. However, these methods often need more structural information on knowledge graphs, prompting the challenge of integrating graph structure knowledge into text-based methodologies. To tackle this issue, we introduce a text-enhanced model with local structure (TEGS) that embeds local graph structure details from the knowledge graph into the text encoder. TEGS integrates <em>k</em>-hop neighbor entity information into the text encoder and employs a decoupled attention mechanism to blend relative position encoding and text semantics. This strategy augments learnable content through graph structure information and mitigates the impact of semantic ambiguity via the decoupled attention mechanism. Experimental findings demonstrate TEGS’s effectiveness at fusing graph structure information, resulting in state-of-the-art performance across three datasets in link prediction tasks. In terms of Hit@1, when compared to the previous text-based models, our model demonstrated improvements of 2.1% on WN18RR, 2.4% on FB15k-237, and 2.7% on the NELL-One dataset. Our code is made publicly available on <span>https://github.com/HubuKG/TEGS</span><svg><path></path></svg>.</p></div>\",\"PeriodicalId\":50365,\"journal\":{\"name\":\"Information Processing & Management\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":7.4000,\"publicationDate\":\"2024-06-11\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Information Processing & Management\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0306457324001560\",\"RegionNum\":1,\"RegionCategory\":\"管理学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"COMPUTER SCIENCE, INFORMATION SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Information Processing & Management","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0306457324001560","RegionNum":1,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0
摘要
知识图谱表征学习需要将知识图谱中的实体和关系转化为向量,以加强下游任务。最近,预训练语言模型的兴起推动了基于文本的知识图谱表示学习方法。然而,这些方法往往需要更多的知识图谱结构信息,这就给将图谱结构知识整合到基于文本的方法中带来了挑战。为了解决这个问题,我们引入了一种具有局部结构的文本增强模型(TEGS),它将知识图谱中的局部图结构细节嵌入到文本编码器中。TEGS 将 k 跳邻居实体信息整合到文本编码器中,并采用解耦注意力机制来融合相对位置编码和文本语义。这一策略通过图结构信息增加了可学习的内容,并通过解耦注意力机制减轻了语义模糊的影响。实验结果证明了 TEGS 在融合图结构信息方面的有效性,在链接预测任务的三个数据集中取得了最先进的性能。在Hit@1方面,与之前基于文本的模型相比,我们的模型在WN18RR上提高了2.1%,在FB15k-237上提高了2.4%,在NELL-One数据集上提高了2.7%。我们的代码可在 https://github.com/HubuKG/TEGS 上公开获取。
Text-enhanced knowledge graph representation learning with local structure
Knowledge graph representation learning entails transforming entities and relationships within a knowledge graph into vectors to enhance downstream tasks. The rise of pre-trained language models has recently promoted text-based approaches for knowledge graph representation learning. However, these methods often need more structural information on knowledge graphs, prompting the challenge of integrating graph structure knowledge into text-based methodologies. To tackle this issue, we introduce a text-enhanced model with local structure (TEGS) that embeds local graph structure details from the knowledge graph into the text encoder. TEGS integrates k-hop neighbor entity information into the text encoder and employs a decoupled attention mechanism to blend relative position encoding and text semantics. This strategy augments learnable content through graph structure information and mitigates the impact of semantic ambiguity via the decoupled attention mechanism. Experimental findings demonstrate TEGS’s effectiveness at fusing graph structure information, resulting in state-of-the-art performance across three datasets in link prediction tasks. In terms of Hit@1, when compared to the previous text-based models, our model demonstrated improvements of 2.1% on WN18RR, 2.4% on FB15k-237, and 2.7% on the NELL-One dataset. Our code is made publicly available on https://github.com/HubuKG/TEGS.
期刊介绍:
Information Processing and Management is dedicated to publishing cutting-edge original research at the convergence of computing and information science. Our scope encompasses theory, methods, and applications across various domains, including advertising, business, health, information science, information technology marketing, and social computing.
We aim to cater to the interests of both primary researchers and practitioners by offering an effective platform for the timely dissemination of advanced and topical issues in this interdisciplinary field. The journal places particular emphasis on original research articles, research survey articles, research method articles, and articles addressing critical applications of research. Join us in advancing knowledge and innovation at the intersection of computing and information science.