Investigating Self-Rationalizing Models for Commonsense Reasoning

IF 0.9 Q4 MATHEMATICS, INTERDISCIPLINARY APPLICATIONS Stats Pub Date : 2023-08-29 DOI:10.3390/stats6030056
Fanny Rancourt, Paula Vondrlik, Diego Maupomé, Marie-Jean Meurs
{"title":"Investigating Self-Rationalizing Models for Commonsense Reasoning","authors":"Fanny Rancourt, Paula Vondrlik, Diego Maupomé, Marie-Jean Meurs","doi":"10.3390/stats6030056","DOIUrl":null,"url":null,"abstract":"The rise of explainable natural language processing spurred a bulk of work on datasets augmented with human explanations, as well as technical approaches to leverage them. Notably, generative large language models offer new possibilities, as they can output a prediction as well as an explanation in natural language. This work investigates the capabilities of fine-tuned text-to-text transfer Transformer (T5) models for commonsense reasoning and explanation generation. Our experiments suggest that while self-rationalizing models achieve interesting results, a significant gap remains: classifiers consistently outperformed self-rationalizing models, and a substantial fraction of model-generated explanations are not valid. Furthermore, training with expressive free-text explanations substantially altered the inner representation of the model, suggesting that they supplied additional information and may bridge the knowledge gap. Our code is publicly available, and the experiments were run on open-access datasets, hence allowing full reproducibility.","PeriodicalId":93142,"journal":{"name":"Stats","volume":" ","pages":""},"PeriodicalIF":0.9000,"publicationDate":"2023-08-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Stats","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.3390/stats6030056","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q4","JCRName":"MATHEMATICS, INTERDISCIPLINARY APPLICATIONS","Score":null,"Total":0}
引用次数: 0

Abstract

The rise of explainable natural language processing spurred a bulk of work on datasets augmented with human explanations, as well as technical approaches to leverage them. Notably, generative large language models offer new possibilities, as they can output a prediction as well as an explanation in natural language. This work investigates the capabilities of fine-tuned text-to-text transfer Transformer (T5) models for commonsense reasoning and explanation generation. Our experiments suggest that while self-rationalizing models achieve interesting results, a significant gap remains: classifiers consistently outperformed self-rationalizing models, and a substantial fraction of model-generated explanations are not valid. Furthermore, training with expressive free-text explanations substantially altered the inner representation of the model, suggesting that they supplied additional information and may bridge the knowledge gap. Our code is publicly available, and the experiments were run on open-access datasets, hence allowing full reproducibility.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
常识推理的自合理化模型研究
可解释的自然语言处理的兴起促使人们在数据集上进行了大量的工作,这些数据集增加了人类的解释,以及利用这些解释的技术方法。值得注意的是,生成性大型语言模型提供了新的可能性,因为它们可以用自然语言输出预测和解释。这项工作研究了微调的文本到文本转换转换器(T5)模型用于常识推理和解释生成的能力。我们的实验表明,虽然自合理化模型取得了令人感兴趣的结果,但仍存在一个显著的差距:分类器始终优于自合理化模式,并且相当一部分模型生成的解释是无效的。此外,用富有表现力的自由文本解释进行训练大大改变了模型的内部表示,这表明它们提供了额外的信息,并可能弥合知识差距。我们的代码是公开的,实验是在开放访问的数据集上运行的,因此允许完全的再现性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
CiteScore
0.60
自引率
0.00%
发文量
0
审稿时长
7 weeks
期刊最新文献
Precise Tensor Product Smoothing via Spectral Splines Predicting Random Walks and a Data-Splitting Prediction Region The Mediating Impact of Innovation Types in the Relationship between Innovation Use Theory and Market Performance Jump-Robust Realized-GARCH-MIDAS-X Estimators for Bitcoin and Ethereum Volatility Indices Revisiting the Large n (Sample Size) Problem: How to Avert Spurious Significance Results
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1