基于语言的常识性问题解答推理图神经网络。

IF 6 1区 计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE Neural Networks Pub Date : 2024-10-18 DOI:10.1016/j.neunet.2024.106816
Meng Yang, Yihao Wang, Yu Gu
{"title":"基于语言的常识性问题解答推理图神经网络。","authors":"Meng Yang,&nbsp;Yihao Wang,&nbsp;Yu Gu","doi":"10.1016/j.neunet.2024.106816","DOIUrl":null,"url":null,"abstract":"<div><div>Language model (LM) has played an increasingly important role in the common-sense understanding and reasoning in the CSQA task (Common Sense Question Answering). However, due to the amount of model parameters, increasing training data helps little in further improving model performance. Introducing external knowledge through graph neural networks (GNNs) proves positive in boosting performance, but exploiting different knowledge sources and capturing contextual information between text and knowledge inside remains a challenge. In this paper, we propose LBR-GNN, a <strong>L</strong>anguage-<strong>B</strong>ased <strong>R</strong>easoning <strong>G</strong>raph <strong>N</strong>eural <strong>N</strong>etwork method to address these problems, by representing the question with each answer and external knowledge using a language model and predicting the reasoning score with a designed language-based GNN. Our LBR-GNN will first regulate external knowledge into a consistent textual form and encode it using a standard LM to capture the contextual information. Then, we build a graph neural network using the encoded information, especially the language-level edge representation. Finally, we design a novel edge aggregation method to select the edge information for GNN update and the language-guided GNN reasoning. We assess the performance of LBR-GNN across the CommonsenseQA, CommonsenseQA-IH, and OpenBookQA datasets. Our evaluation reveals a performance boost of more than 5% compared to the state-of-the-art methods on the CSQA dataset, achieved with a similar number of additional parameters.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"181 ","pages":"Article 106816"},"PeriodicalIF":6.0000,"publicationDate":"2024-10-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Language-based reasoning graph neural network for commonsense question answering\",\"authors\":\"Meng Yang,&nbsp;Yihao Wang,&nbsp;Yu Gu\",\"doi\":\"10.1016/j.neunet.2024.106816\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>Language model (LM) has played an increasingly important role in the common-sense understanding and reasoning in the CSQA task (Common Sense Question Answering). However, due to the amount of model parameters, increasing training data helps little in further improving model performance. Introducing external knowledge through graph neural networks (GNNs) proves positive in boosting performance, but exploiting different knowledge sources and capturing contextual information between text and knowledge inside remains a challenge. In this paper, we propose LBR-GNN, a <strong>L</strong>anguage-<strong>B</strong>ased <strong>R</strong>easoning <strong>G</strong>raph <strong>N</strong>eural <strong>N</strong>etwork method to address these problems, by representing the question with each answer and external knowledge using a language model and predicting the reasoning score with a designed language-based GNN. Our LBR-GNN will first regulate external knowledge into a consistent textual form and encode it using a standard LM to capture the contextual information. Then, we build a graph neural network using the encoded information, especially the language-level edge representation. Finally, we design a novel edge aggregation method to select the edge information for GNN update and the language-guided GNN reasoning. We assess the performance of LBR-GNN across the CommonsenseQA, CommonsenseQA-IH, and OpenBookQA datasets. Our evaluation reveals a performance boost of more than 5% compared to the state-of-the-art methods on the CSQA dataset, achieved with a similar number of additional parameters.</div></div>\",\"PeriodicalId\":49763,\"journal\":{\"name\":\"Neural Networks\",\"volume\":\"181 \",\"pages\":\"Article 106816\"},\"PeriodicalIF\":6.0000,\"publicationDate\":\"2024-10-18\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Neural Networks\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0893608024007408\",\"RegionNum\":1,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Neural Networks","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0893608024007408","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0

摘要

语言模型(LM)在 CSQA 任务(常识问题解答)的常识理解和推理中发挥着越来越重要的作用。然而,由于模型参数量大,增加训练数据对进一步提高模型性能帮助不大。事实证明,通过图神经网络(GNN)引入外部知识对提高性能有积极作用,但如何利用不同的知识源并捕捉文本与内部知识之间的上下文信息仍是一个挑战。本文提出了一种基于语言的推理图神经网络(LBR-GNN)方法来解决这些问题,即用语言模型表示问题、每个答案和外部知识,并用设计的基于语言的图神经网络预测推理得分。我们的 LBR-GNN 首先会将外部知识规范为一致的文本形式,并使用标准 LM 对其进行编码,以捕捉上下文信息。然后,我们利用编码信息,尤其是语言级边缘表示,构建图神经网络。最后,我们设计了一种新颖的边缘聚合方法,用于为 GNN 更新和语言引导的 GNN 推理选择边缘信息。我们在 CommonsenseQA、CommonsenseQA-IH 和 OpenBookQA 数据集上评估了 LBR-GNN 的性能。我们的评估结果表明,与 CSQA 数据集上的先进方法相比,LBR-GNN 的性能提高了 5%以上,而且是在附加参数数量相近的情况下实现的。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
Language-based reasoning graph neural network for commonsense question answering
Language model (LM) has played an increasingly important role in the common-sense understanding and reasoning in the CSQA task (Common Sense Question Answering). However, due to the amount of model parameters, increasing training data helps little in further improving model performance. Introducing external knowledge through graph neural networks (GNNs) proves positive in boosting performance, but exploiting different knowledge sources and capturing contextual information between text and knowledge inside remains a challenge. In this paper, we propose LBR-GNN, a Language-Based Reasoning Graph Neural Network method to address these problems, by representing the question with each answer and external knowledge using a language model and predicting the reasoning score with a designed language-based GNN. Our LBR-GNN will first regulate external knowledge into a consistent textual form and encode it using a standard LM to capture the contextual information. Then, we build a graph neural network using the encoded information, especially the language-level edge representation. Finally, we design a novel edge aggregation method to select the edge information for GNN update and the language-guided GNN reasoning. We assess the performance of LBR-GNN across the CommonsenseQA, CommonsenseQA-IH, and OpenBookQA datasets. Our evaluation reveals a performance boost of more than 5% compared to the state-of-the-art methods on the CSQA dataset, achieved with a similar number of additional parameters.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Neural Networks
Neural Networks 工程技术-计算机:人工智能
CiteScore
13.90
自引率
7.70%
发文量
425
审稿时长
67 days
期刊介绍: Neural Networks is a platform that aims to foster an international community of scholars and practitioners interested in neural networks, deep learning, and other approaches to artificial intelligence and machine learning. Our journal invites submissions covering various aspects of neural networks research, from computational neuroscience and cognitive modeling to mathematical analyses and engineering applications. By providing a forum for interdisciplinary discussions between biology and technology, we aim to encourage the development of biologically-inspired artificial intelligence.
期刊最新文献
Multi-source Selective Graph Domain Adaptation Network for cross-subject EEG emotion recognition. Spectral integrated neural networks (SINNs) for solving forward and inverse dynamic problems. Corrigendum to “Multi-view Graph Pooling with Coarsened Graph Disentanglement” [Neural Networks 174 (2024) 1-10/106221] Rethinking the impact of noisy labels in graph classification: A utility and privacy perspective Multilevel semantic and adaptive actionness learning for weakly supervised temporal action localization
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1