首页 > 最新文献

Proceedings of COLING. International Conference on Computational Linguistics最新文献

英文 中文
Reinforcement Learning with Large Action Spaces for Neural Machine Translation 基于大动作空间的神经机器翻译强化学习
Pub Date : 2022-10-06 DOI: 10.48550/arXiv.2210.03053
Asaf Yehudai, Leshem Choshen, Lior Fox, Omri Abend
Applying Reinforcement learning (RL) following maximum likelihood estimation (MLE) pre-training is a versatile method for enhancing neural machine translation (NMT) performance. However, recent work has argued that the gains produced by RL for NMT are mostly due to promoting tokens that have already received a fairly high probability in pre-training. We hypothesize that the large action space is a main obstacle to RL’s effectiveness in MT, and conduct two sets of experiments that lend support to our hypothesis. First, we find that reducing the size of the vocabulary improves RL’s effectiveness. Second, we find that effectively reducing the dimension of the action space without changing the vocabulary also yields notable improvement as evaluated by BLEU, semantic similarity, and human evaluation. Indeed, by initializing the network’s final fully connected layer (that maps the network’s internal dimension to the vocabulary dimension), with a layer that generalizes over similar actions, we obtain a substantial improvement in RL performance: 1.5 BLEU points on average.
在极大似然估计(MLE)预训练之后应用强化学习(RL)是提高神经机器翻译(NMT)性能的一种通用方法。然而,最近的研究认为,RL为NMT产生的收益主要是由于在预训练中已经获得相当高概率的推广令牌。我们假设大的动作空间是RL在MT中的有效性的主要障碍,并进行了两组实验来支持我们的假设。首先,我们发现减少词汇量可以提高强化学习的有效性。其次,我们发现,在不改变词汇量的情况下,有效地降低动作空间的维数也会产生显著的改进,如BLEU、语义相似度和人类评价。事实上,通过初始化网络的最终全连接层(将网络的内部维度映射到词汇表维度),并使用一个泛化类似动作的层,我们在RL性能上获得了实质性的改进:平均1.5 BLEU点。
{"title":"Reinforcement Learning with Large Action Spaces for Neural Machine Translation","authors":"Asaf Yehudai, Leshem Choshen, Lior Fox, Omri Abend","doi":"10.48550/arXiv.2210.03053","DOIUrl":"https://doi.org/10.48550/arXiv.2210.03053","url":null,"abstract":"Applying Reinforcement learning (RL) following maximum likelihood estimation (MLE) pre-training is a versatile method for enhancing neural machine translation (NMT) performance. However, recent work has argued that the gains produced by RL for NMT are mostly due to promoting tokens that have already received a fairly high probability in pre-training. We hypothesize that the large action space is a main obstacle to RL’s effectiveness in MT, and conduct two sets of experiments that lend support to our hypothesis. First, we find that reducing the size of the vocabulary improves RL’s effectiveness. Second, we find that effectively reducing the dimension of the action space without changing the vocabulary also yields notable improvement as evaluated by BLEU, semantic similarity, and human evaluation. Indeed, by initializing the network’s final fully connected layer (that maps the network’s internal dimension to the vocabulary dimension), with a layer that generalizes over similar actions, we obtain a substantial improvement in RL performance: 1.5 BLEU points on average.","PeriodicalId":91381,"journal":{"name":"Proceedings of COLING. International Conference on Computational Linguistics","volume":"109 1","pages":"4544-4556"},"PeriodicalIF":0.0,"publicationDate":"2022-10-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82061459","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
Teaching Neural Module Networks to Do Arithmetic 教神经模块网络做算术
Pub Date : 2022-10-06 DOI: 10.48550/arXiv.2210.02703
Jiayi Chen, Xiao-Yu Guo, Yuan-Fang Li, Gholamreza Haffari
Answering complex questions that require multi-step multi-type reasoning over raw text is challenging, especially when conducting numerical reasoning. Neural Module Networks (NMNs), follow the programmer-interpreter framework and design trainable modules to learn different reasoning skills. However, NMNs only have limited reasoning abilities, and lack numerical reasoning capability. We upgrade NMNs by: (a) bridging the gap between its interpreter and the complex questions; (b) introducing addition and subtraction modules that perform numerical reasoning over numbers. On a subset of DROP, experimental results show that our proposed methods enhance NMNs’ numerical reasoning skills by 17.7% improvement of F1 score and significantly outperform previous state-of-the-art models.
回答需要对原始文本进行多步骤多类型推理的复杂问题是具有挑战性的,特别是在进行数值推理时。神经模块网络(NMNs),遵循程序员-解释器框架,设计可训练的模块来学习不同的推理技能。然而,神经网络的推理能力有限,缺乏数值推理能力。我们通过以下方式升级神经网络:(a)弥合其解释器与复杂问题之间的差距;(b)引入对数字进行数值推理的加法和减法模块。在DROP的一个子集上,实验结果表明,我们提出的方法使NMNs的数值推理能力提高了17.7%,并且显著优于以前最先进的模型。
{"title":"Teaching Neural Module Networks to Do Arithmetic","authors":"Jiayi Chen, Xiao-Yu Guo, Yuan-Fang Li, Gholamreza Haffari","doi":"10.48550/arXiv.2210.02703","DOIUrl":"https://doi.org/10.48550/arXiv.2210.02703","url":null,"abstract":"Answering complex questions that require multi-step multi-type reasoning over raw text is challenging, especially when conducting numerical reasoning. Neural Module Networks (NMNs), follow the programmer-interpreter framework and design trainable modules to learn different reasoning skills. However, NMNs only have limited reasoning abilities, and lack numerical reasoning capability. We upgrade NMNs by: (a) bridging the gap between its interpreter and the complex questions; (b) introducing addition and subtraction modules that perform numerical reasoning over numbers. On a subset of DROP, experimental results show that our proposed methods enhance NMNs’ numerical reasoning skills by 17.7% improvement of F1 score and significantly outperform previous state-of-the-art models.","PeriodicalId":91381,"journal":{"name":"Proceedings of COLING. International Conference on Computational Linguistics","volume":"24 1","pages":"1502-1510"},"PeriodicalIF":0.0,"publicationDate":"2022-10-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83833021","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Debiasing Isn’t Enough! – on the Effectiveness of Debiasing MLMs and Their Social Biases in Downstream Tasks 消除偏见是不够的!——关于消除传销及其社会偏见在下游任务中的有效性
Pub Date : 2022-10-06 DOI: 10.48550/arXiv.2210.02938
Masahiro Kaneko, D. Bollegala, Naoaki Okazaki
We study the relationship between task-agnostic intrinsic and task-specific extrinsic social bias evaluation measures for MLMs, and find that there exists only a weak correlation between these two types of evaluation measures. Moreover, we find that MLMs debiased using different methods still re-learn social biases during fine-tuning on downstream tasks. We identify the social biases in both training instances as well as their assigned labels as reasons for the discrepancy between intrinsic and extrinsic bias evaluation measurements. Overall, our findings highlight the limitations of existing MLM bias evaluation measures and raise concerns on the deployment of MLMs in downstream applications using those measures.
我们研究了传销商任务不可知论的内在社会偏见和任务特定的外在社会偏见评价指标之间的关系,发现这两种评价指标之间仅存在弱相关。此外,我们发现使用不同方法去偏见的传销在下游任务的微调过程中仍然会重新学习社会偏见。我们确定了两个训练实例中的社会偏见以及它们分配的标签,作为内在和外在偏见评估测量结果之间差异的原因。总的来说,我们的研究结果强调了现有的传销偏见评估措施的局限性,并提出了对传销在下游应用中使用这些措施的关注。
{"title":"Debiasing Isn’t Enough! – on the Effectiveness of Debiasing MLMs and Their Social Biases in Downstream Tasks","authors":"Masahiro Kaneko, D. Bollegala, Naoaki Okazaki","doi":"10.48550/arXiv.2210.02938","DOIUrl":"https://doi.org/10.48550/arXiv.2210.02938","url":null,"abstract":"We study the relationship between task-agnostic intrinsic and task-specific extrinsic social bias evaluation measures for MLMs, and find that there exists only a weak correlation between these two types of evaluation measures. Moreover, we find that MLMs debiased using different methods still re-learn social biases during fine-tuning on downstream tasks. We identify the social biases in both training instances as well as their assigned labels as reasons for the discrepancy between intrinsic and extrinsic bias evaluation measurements. Overall, our findings highlight the limitations of existing MLM bias evaluation measures and raise concerns on the deployment of MLMs in downstream applications using those measures.","PeriodicalId":91381,"journal":{"name":"Proceedings of COLING. International Conference on Computational Linguistics","volume":"148 1","pages":"1299-1310"},"PeriodicalIF":0.0,"publicationDate":"2022-10-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77857391","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 18
Schema Encoding for Transferable Dialogue State Tracking 可转移对话状态跟踪的模式编码
Pub Date : 2022-10-05 DOI: 10.48550/arXiv.2210.02351
Hyunmin Jeon, G. G. Lee
Dialogue state tracking (DST) is an essential sub-task for task-oriented dialogue systems. Recent work has focused on deep neural models for DST. However, the neural models require a large dataset for training. Furthermore, applying them to another domain needs a new dataset because the neural models are generally trained to imitate the given dataset. In this paper, we propose Schema Encoding for Transferable Dialogue State Tracking (SET-DST), which is a neural DST method for effective transfer to new domains. Transferable DST could assist developments of dialogue systems even with few dataset on target domains. We use a schema encoder not just to imitate the dataset but to comprehend the schema of the dataset. We aim to transfer the model to new domains by encoding new schemas and using them for DST on multi-domain settings. As a result, SET-DST improved the joint accuracy by 1.46 points on MultiWOZ 2.1.
对话状态跟踪(DST)是面向任务的对话系统的重要子任务。最近的工作集中在DST的深度神经模型上。然而,神经模型需要大量的数据集进行训练。此外,将它们应用到另一个领域需要一个新的数据集,因为神经模型通常被训练成模仿给定的数据集。本文提出了基于模式编码的可转移对话状态跟踪(SET-DST)方法,这是一种有效转移到新域的神经DST方法。即使目标域上的数据集很少,可转移的DST也可以帮助开发对话系统。我们使用模式编码器不仅是为了模仿数据集,而且是为了理解数据集的模式。我们的目标是通过编码新模式并将其用于多域设置的DST,将模型转移到新域。因此,SET-DST在MultiWOZ 2.1上将关节精度提高了1.46点。
{"title":"Schema Encoding for Transferable Dialogue State Tracking","authors":"Hyunmin Jeon, G. G. Lee","doi":"10.48550/arXiv.2210.02351","DOIUrl":"https://doi.org/10.48550/arXiv.2210.02351","url":null,"abstract":"Dialogue state tracking (DST) is an essential sub-task for task-oriented dialogue systems. Recent work has focused on deep neural models for DST. However, the neural models require a large dataset for training. Furthermore, applying them to another domain needs a new dataset because the neural models are generally trained to imitate the given dataset. In this paper, we propose Schema Encoding for Transferable Dialogue State Tracking (SET-DST), which is a neural DST method for effective transfer to new domains. Transferable DST could assist developments of dialogue systems even with few dataset on target domains. We use a schema encoder not just to imitate the dataset but to comprehend the schema of the dataset. We aim to transfer the model to new domains by encoding new schemas and using them for DST on multi-domain settings. As a result, SET-DST improved the joint accuracy by 1.46 points on MultiWOZ 2.1.","PeriodicalId":91381,"journal":{"name":"Proceedings of COLING. International Conference on Computational Linguistics","volume":"39 1","pages":"355-366"},"PeriodicalIF":0.0,"publicationDate":"2022-10-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"86489962","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
CorefDiffs: Co-referential and Differential Knowledge Flow in Document Grounded Conversations CorefDiffs:基于文档的对话中的共同参考和差异知识流
Pub Date : 2022-10-05 DOI: 10.48550/arXiv.2210.02223
Lin Xu, Qixian Zhou, Jinlan Fu, Min-Yen Kan, See-Kiong Ng
Knowledge-grounded dialog systems need to incorporate smooth transitions among knowledge selected for generating responses, to ensure that dialog flows naturally. For document-grounded dialog systems, the inter- and intra-document knowledge relations can be used to model such conversational flows. We develop a novel Multi-Document Co-Referential Graph (Coref-MDG) to effectively capture the inter-document relationships based on commonsense and similarity and the intra-document co-referential structures of knowledge segments within the grounding documents. We propose CorefDiffs, a Co-referential and Differential flow management method, to linearize the static Coref-MDG into conversational sequence logic. CorefDiffs performs knowledge selection by accounting for contextual graph structures and the knowledge difference sequences. CorefDiffs significantly outperforms the state-of-the-art by 9.5%, 7.4% and 8.2% on three public benchmarks. This demonstrates that the effective modeling of co-reference and knowledge difference for dialog flows are critical for transitions in document-grounded conversation.
以知识为基础的对话系统需要在为生成响应而选择的知识之间进行平滑转换,以确保对话自然地流动。对于基于文档的对话系统,可以使用文档间和文档内的知识关系来建模这样的会话流。我们开发了一种新的基于常识和相似性的多文档协同引用图(Coref-MDG),以有效地捕获基于基础文档的知识段的文档间关系和文档内的协同引用结构。我们提出CorefDiffs,一种共同引用和差分流管理方法,将静态corefmdg线性化为会话序列逻辑。CorefDiffs通过考虑上下文图结构和知识差异序列来进行知识选择。在三个公开基准测试中,CorefDiffs的表现分别为9.5%、7.4%和8.2%。这表明对话流的共同引用和知识差异的有效建模对于基于文档的对话中的转换是至关重要的。
{"title":"CorefDiffs: Co-referential and Differential Knowledge Flow in Document Grounded Conversations","authors":"Lin Xu, Qixian Zhou, Jinlan Fu, Min-Yen Kan, See-Kiong Ng","doi":"10.48550/arXiv.2210.02223","DOIUrl":"https://doi.org/10.48550/arXiv.2210.02223","url":null,"abstract":"Knowledge-grounded dialog systems need to incorporate smooth transitions among knowledge selected for generating responses, to ensure that dialog flows naturally. For document-grounded dialog systems, the inter- and intra-document knowledge relations can be used to model such conversational flows. We develop a novel Multi-Document Co-Referential Graph (Coref-MDG) to effectively capture the inter-document relationships based on commonsense and similarity and the intra-document co-referential structures of knowledge segments within the grounding documents. We propose CorefDiffs, a Co-referential and Differential flow management method, to linearize the static Coref-MDG into conversational sequence logic. CorefDiffs performs knowledge selection by accounting for contextual graph structures and the knowledge difference sequences. CorefDiffs significantly outperforms the state-of-the-art by 9.5%, 7.4% and 8.2% on three public benchmarks. This demonstrates that the effective modeling of co-reference and knowledge difference for dialog flows are critical for transitions in document-grounded conversation.","PeriodicalId":91381,"journal":{"name":"Proceedings of COLING. International Conference on Computational Linguistics","volume":"33 1","pages":"471-484"},"PeriodicalIF":0.0,"publicationDate":"2022-10-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"80050100","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
A New Public Corpus for Clinical Section Identification: MedSecId. 一个新的临床切片识别公共语料库:MedSecId。
Paul Landes, Kunal Patel, Sean S Huang, Adam Webb, Barbara Di Eugenio, Cornelia Caragea

The process by which sections in a document are demarcated and labeled is known as section identification. Such sections are helpful to the reader when searching for information and contextualizing specific topics. The goal of this work is to segment the sections of clinical medical domain documentation. The primary contribution of this work is MedSecId, a publicly available set of 2,002 fully annotated medical notes from the MIMIC-III. We include several baselines, source code, a pretrained model and analysis of the data showing a relationship between medical concepts across sections using principal component analysis.

在文档中划分和标记各节的过程称为节标识。这些部分有助于读者搜索信息和将特定主题置于上下文环境中。这项工作的目标是分割临床医学领域文档的各个部分。这项工作的主要贡献是MedSecId,这是一套公开的2002份完整注释的医学笔记,来自MIMIC-III。我们包括几个基线、源代码、一个预训练模型和数据分析,使用主成分分析显示医学概念之间的关系。
{"title":"A New Public Corpus for Clinical Section Identification: MedSecId.","authors":"Paul Landes, Kunal Patel, Sean S Huang, Adam Webb, Barbara Di Eugenio, Cornelia Caragea","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>The process by which sections in a document are demarcated and labeled is known as section identification. Such sections are helpful to the reader when searching for information and contextualizing specific topics. The goal of this work is to segment the sections of clinical medical domain documentation. The primary contribution of this work is MedSecId, a publicly available set of 2,002 fully annotated medical notes from the MIMIC-III. We include several baselines, source code, a pretrained model and analysis of the data showing a relationship between medical concepts across sections using principal component analysis.</p>","PeriodicalId":91381,"journal":{"name":"Proceedings of COLING. International Conference on Computational Linguistics","volume":"2022 ","pages":"3709-3721"},"PeriodicalIF":0.0,"publicationDate":"2022-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11627044/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142803934","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Edinburgh_UCL_Health@SMM4H'22: From Glove to Flair for handling imbalanced healthcare corpora related to Adverse Drug Events, Change in medication and self-reporting vaccination. Edinburgh_UCL_Health@SMM4H'22:从手套到Flair处理与药物不良事件,药物变化和自我报告疫苗接种相关的不平衡保健语料库。
Imane Guellil, Jinge Wu, Honghan Wu, Tony Sun, Beatrice Alex

This paper reports on the performance of Edin-burgh_UCL_Health's models in the Social Media Mining for Health (SMM4H) 2022 shared tasks. Our team participated in the tasks related to the Identification of Adverse Drug Events (ADEs), the classification of change in medication (change-med) and the classification of selfreport of vaccination (self-vaccine). Our best performing models are based on DeepADEM-iner (with respective F1= 0.64, 0.62 and 0.39 for ADE identification), on a GloVe model trained on Twitter (with F1=0.11 for the changemed) and finally on a stack embedding including a layer of Glove embedding and two layers of Flair embedding (with F1= 0.77 for selfreport).

本文报告了Edin-burgh_UCL_Health模型在社交媒体挖掘健康(SMM4H) 2022共享任务中的性能。我们小组参与了药物不良事件识别(ADEs)、药物变化分类(change-med)和疫苗接种自我报告分类(self-vaccine)的相关任务。我们表现最好的模型是基于DeepADEM-iner (ADE识别的F1分别为0.64、0.62和0.39),基于在Twitter上训练的GloVe模型(对于changemed, F1=0.11),最后是基于一个堆栈嵌入,包括一层GloVe嵌入和两层Flair嵌入(self - port, F1= 0.77)。
{"title":"Edinburgh_UCL_Health@SMM4H'22: From Glove to Flair for handling imbalanced healthcare corpora related to Adverse Drug Events, Change in medication and self-reporting vaccination.","authors":"Imane Guellil,&nbsp;Jinge Wu,&nbsp;Honghan Wu,&nbsp;Tony Sun,&nbsp;Beatrice Alex","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>This paper reports on the performance of Edin-burgh_UCL_Health's models in the Social Media Mining for Health (SMM4H) 2022 shared tasks. Our team participated in the tasks related to the Identification of Adverse Drug Events (ADEs), the classification of change in medication (change-med) and the classification of selfreport of vaccination (self-vaccine). Our best performing models are based on DeepADEM-iner (with respective F1= 0.64, 0.62 and 0.39 for ADE identification), on a GloVe model trained on Twitter (with F1=0.11 for the changemed) and finally on a stack embedding including a layer of Glove embedding and two layers of Flair embedding (with F1= 0.77 for selfreport).</p>","PeriodicalId":91381,"journal":{"name":"Proceedings of COLING. International Conference on Computational Linguistics","volume":"2022 ","pages":"148-152"},"PeriodicalIF":0.0,"publicationDate":"2022-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7613791/pdf/EMS156318.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"40683831","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Evaluating the Performance of Transformer-based Language Models for Neuroatypical Language. 基于转换器的神经非典型语言模型的性能评估。
Duanchen Liu, Zoey Liu, Qingyun Yang, Yujing Huang, Emily Prud'hommeaux

Difficulties with social aspects of language are among the hallmarks of autism spectrum disorder (ASD). These communication differences are thought to contribute to the challenges that adults with ASD experience when seeking employment, underscoring the need for interventions that focus on improving areas of weakness in pragmatic and social language. In this paper, we describe a transformer-based framework for identifying linguistic features associated with social aspects of communication using a corpus of conversations between adults with and without ASD and neurotypical conversational partners produced while engaging in collaborative tasks. While our framework yields strong accuracy overall, performance is significantly worse for the language of participants with ASD, suggesting that they use a more diverse set of strategies for some social linguistic functions. These results, while showing promise for the development of automated language analysis tools to support targeted language interventions for ASD, also reveal weaknesses in the ability of large contextualized language models to model neuroatypical language.

语言社交方面的困难是自闭症谱系障碍(ASD)的特征之一。这些沟通差异被认为是导致自闭症成年人在找工作时遇到挑战的原因,强调需要采取干预措施,重点改善实用主义和社交语言方面的弱点。在本文中,我们描述了一个基于转换的框架,用于识别与沟通的社会方面相关的语言特征,该框架使用的是在参与协作任务时产生的有或没有ASD的成年人与神经正常对话伙伴之间的对话语料库。虽然我们的框架总体上具有很强的准确性,但自闭症谱系障碍参与者的语言表现明显更差,这表明他们在某些社会语言功能上使用了一套更多样化的策略。这些结果,虽然显示了自动化语言分析工具的发展前景,以支持针对ASD的有针对性的语言干预,但也揭示了大型情境化语言模型模拟神经非典型语言能力的弱点。
{"title":"Evaluating the Performance of Transformer-based Language Models for Neuroatypical Language.","authors":"Duanchen Liu,&nbsp;Zoey Liu,&nbsp;Qingyun Yang,&nbsp;Yujing Huang,&nbsp;Emily Prud'hommeaux","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Difficulties with social aspects of language are among the hallmarks of autism spectrum disorder (ASD). These communication differences are thought to contribute to the challenges that adults with ASD experience when seeking employment, underscoring the need for interventions that focus on improving areas of weakness in pragmatic and social language. In this paper, we describe a transformer-based framework for identifying linguistic features associated with social aspects of communication using a corpus of conversations between adults with and without ASD and neurotypical conversational partners produced while engaging in collaborative tasks. While our framework yields strong accuracy overall, performance is significantly worse for the language of participants with ASD, suggesting that they use a more diverse set of strategies for some social linguistic functions. These results, while showing promise for the development of automated language analysis tools to support targeted language interventions for ASD, also reveal weaknesses in the ability of large contextualized language models to model neuroatypical language.</p>","PeriodicalId":91381,"journal":{"name":"Proceedings of COLING. International Conference on Computational Linguistics","volume":"2022 ","pages":"3412-3419"},"PeriodicalIF":0.0,"publicationDate":"2022-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9633182/pdf/nihms-1846175.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"40683834","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Summarizing Patients' Problems from Hospital Progress Notes Using Pre-trained Sequence-to-Sequence Models. 使用预先训练的序列到序列模型总结医院进展记录中的患者问题。
Yanjun Gao, Timothy Miller, Dongfang Xu, Dmitriy Dligach, Matthew M Churpek, Majid Afshar

Automatically summarizing patients' main problems from daily progress notes using natural language processing methods helps to battle against information and cognitive overload in hospital settings and potentially assists providers with computerized diagnostic decision support. Problem list summarization requires a model to understand, abstract, and generate clinical documentation. In this work, we propose a new NLP task that aims to generate a list of problems in a patient's daily care plan using input from the provider's progress notes during hospitalization. We investigate the performance of T5 and BART, two state-of-the-art seq2seq transformer architectures, in solving this problem. We provide a corpus built on top of progress notes from publicly available electronic health record progress notes in the Medical Information Mart for Intensive Care (MIMIC)-III. T5 and BART are trained on general domain text, and we experiment with a data augmentation method and a domain adaptation pre-training method to increase exposure to medical vocabulary and knowledge. Evaluation methods include ROUGE, BERTScore, cosine similarity on sentence embedding, and F-score on medical concepts. Results show that T5 with domain adaptive pre-training achieves significant performance gains compared to a rule-based system and general domain pre-trained language models, indicating a promising direction for tackling the problem summarization task.

使用自然语言处理方法从每日病程记录中自动总结患者的主要问题,有助于对抗医院环境中的信息和认知超载,并可能帮助提供者提供计算机化的诊断决策支持。问题列表总结需要一个模型来理解、抽象和生成临床文档。在这项工作中,我们提出了一个新的NLP任务,旨在使用住院期间提供者的进度记录输入生成患者日常护理计划中的问题列表。我们研究了T5和BART这两种最先进的seq2seq变压器架构在解决这个问题方面的性能。我们提供了一个基于重症监护医疗信息市场(MIMIC)-III中公开可用的电子健康记录进度记录的语料库。T5和BART在一般领域文本上进行训练,我们采用数据增强方法和领域适应预训练方法来增加医学词汇和知识的接触。评价方法包括ROUGE、BERTScore、句子嵌入余弦相似度和医学概念f分。结果表明,与基于规则的系统和一般的领域预训练语言模型相比,具有领域自适应预训练的T5取得了显著的性能提升,为解决问题摘要任务指明了一个有希望的方向。
{"title":"Summarizing Patients' Problems from Hospital Progress Notes Using Pre-trained Sequence-to-Sequence Models.","authors":"Yanjun Gao,&nbsp;Timothy Miller,&nbsp;Dongfang Xu,&nbsp;Dmitriy Dligach,&nbsp;Matthew M Churpek,&nbsp;Majid Afshar","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Automatically summarizing patients' main problems from daily progress notes using natural language processing methods helps to battle against information and cognitive overload in hospital settings and potentially assists providers with computerized diagnostic decision support. Problem list summarization requires a model to understand, abstract, and generate clinical documentation. In this work, we propose a new NLP task that aims to generate a list of problems in a patient's daily care plan using input from the provider's progress notes during hospitalization. We investigate the performance of T5 and BART, two state-of-the-art seq2seq transformer architectures, in solving this problem. We provide a corpus built on top of progress notes from publicly available electronic health record progress notes in the Medical Information Mart for Intensive Care (MIMIC)-III. T5 and BART are trained on general domain text, and we experiment with a data augmentation method and a domain adaptation pre-training method to increase exposure to medical vocabulary and knowledge. Evaluation methods include ROUGE, BERTScore, cosine similarity on sentence embedding, and F-score on medical concepts. Results show that T5 with domain adaptive pre-training achieves significant performance gains compared to a rule-based system and general domain pre-trained language models, indicating a promising direction for tackling the problem summarization task.</p>","PeriodicalId":91381,"journal":{"name":"Proceedings of COLING. International Conference on Computational Linguistics","volume":"2022 ","pages":"2979-2991"},"PeriodicalIF":0.0,"publicationDate":"2022-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9581107/pdf/nihms-1840629.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"40648441","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Medical Question Understanding and Answering with Knowledge Grounding and Semantic Self-Supervision 基于知识基础和语义自我监督的医学问题理解与回答
Pub Date : 2022-09-30 DOI: 10.48550/arXiv.2209.15301
Khalil Mrini, Harpreet Singh, Franck Dernoncourt, Seunghyun Yoon, Trung Bui, Walter Chang, E. Farcas, Ndapandula Nakashole
Current medical question answering systems have difficulty processing long, detailed and informally worded questions submitted by patients, called Consumer Health Questions (CHQs). To address this issue, we introduce a medical question understanding and answering system with knowledge grounding and semantic self-supervision. Our system is a pipeline that first summarizes a long, medical, user-written question, using a supervised summarization loss. Then, our system performs a two-step retrieval to return answers. The system first matches the summarized user question with an FAQ from a trusted medical knowledge base, and then retrieves a fixed number of relevant sentences from the corresponding answer document. In the absence of labels for question matching or answer relevance, we design 3 novel, self-supervised and semantically-guided losses. We evaluate our model against two strong retrieval-based question answering baselines. Evaluators ask their own questions and rate the answers retrieved by our baselines and own system according to their relevance. They find that our system retrieves more relevant answers, while achieving speeds 20 times faster. Our self-supervised losses also help the summarizer achieve higher scores in ROUGE, as well as in human evaluation metrics.
目前的医疗问答系统在处理被称为消费者健康问题(CHQs)的病人提交的冗长、详细且措辞非正式的问题时存在困难。为了解决这一问题,我们引入了一个具有知识基础和语义自我监督的医学问题理解与回答系统。我们的系统是一个管道,首先使用有监督的摘要丢失来总结一个长的、医学的、用户编写的问题。然后,我们的系统执行两步检索来返回答案。系统首先将汇总的用户问题与可信医学知识库中的FAQ进行匹配,然后从相应的答案文档中检索固定数量的相关句子。在缺乏问题匹配或答案相关性标签的情况下,我们设计了3种新颖的、自我监督的和语义引导的损失。我们根据两个强大的基于检索的问答基线来评估我们的模型。评估者提出他们自己的问题,并根据他们的相关性对我们的基线和自己的系统检索到的答案进行评级。他们发现,我们的系统检索到更多相关的答案,同时速度提高了20倍。我们的自我监督损失也帮助总结器在ROUGE以及人类评估指标中获得更高的分数。
{"title":"Medical Question Understanding and Answering with Knowledge Grounding and Semantic Self-Supervision","authors":"Khalil Mrini, Harpreet Singh, Franck Dernoncourt, Seunghyun Yoon, Trung Bui, Walter Chang, E. Farcas, Ndapandula Nakashole","doi":"10.48550/arXiv.2209.15301","DOIUrl":"https://doi.org/10.48550/arXiv.2209.15301","url":null,"abstract":"Current medical question answering systems have difficulty processing long, detailed and informally worded questions submitted by patients, called Consumer Health Questions (CHQs). To address this issue, we introduce a medical question understanding and answering system with knowledge grounding and semantic self-supervision. Our system is a pipeline that first summarizes a long, medical, user-written question, using a supervised summarization loss. Then, our system performs a two-step retrieval to return answers. The system first matches the summarized user question with an FAQ from a trusted medical knowledge base, and then retrieves a fixed number of relevant sentences from the corresponding answer document. In the absence of labels for question matching or answer relevance, we design 3 novel, self-supervised and semantically-guided losses. We evaluate our model against two strong retrieval-based question answering baselines. Evaluators ask their own questions and rate the answers retrieved by our baselines and own system according to their relevance. They find that our system retrieves more relevant answers, while achieving speeds 20 times faster. Our self-supervised losses also help the summarizer achieve higher scores in ROUGE, as well as in human evaluation metrics.","PeriodicalId":91381,"journal":{"name":"Proceedings of COLING. International Conference on Computational Linguistics","volume":"31 1","pages":"2734-2747"},"PeriodicalIF":0.0,"publicationDate":"2022-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"75078118","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
期刊
Proceedings of COLING. International Conference on Computational Linguistics
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1