James Britton, Yan Cong, Yu-Yin Hsu, Emmanuele Chersoni, Philippe Blache
{"title":"论话语连接词对人类和语言模型预测的影响。","authors":"James Britton, Yan Cong, Yu-Yin Hsu, Emmanuele Chersoni, Philippe Blache","doi":"10.3389/fnhum.2024.1363120","DOIUrl":null,"url":null,"abstract":"<p><p>Psycholinguistic literature has consistently shown that humans rely on a rich and organized understanding of event knowledge to predict the forthcoming linguistic input during online sentence comprehension. We, the authors, expect sentences to maintain coherence with the preceding context, making congruent sentence sequences easier to process than incongruent ones. It is widely known that discourse relations between sentences (e.g., temporal, contingency, comparison) are generally made explicit through specific particles, known as <i>discourse connectives</i>, (e.g., <i>and, but, because, after</i>). However, some relations that are easily accessible to the speakers, given their event knowledge, can also be left implicit. The goal of this paper is to investigate the importance of discourse connectives in the prediction of events in human language processing and pretrained language models, with a specific focus on concessives and contrastives, which signal to comprehenders that their event-related predictions have to be <i>reversed</i>. Inspired by previous work, we built a comprehensive set of story stimuli in Italian and Mandarin Chinese that differ in the plausibility and coherence of the situation being described and the presence or absence of a discourse connective. We collected plausibility judgments and reading times from native speakers for the stimuli. Moreover, we correlated the results of the experiments with the predictions given by computational modeling, using Surprisal scores obtained via Transformer-based language models. The human judgements were collected using a seven-point Likert scale and analyzed using cumulative link mixed modeling (CLMM), while the human reading times and language model surprisal scores were analyzed using linear mixed effects regression (LMER). We found that Chinese NLMs are sensitive to plausibility and connectives, although they struggle to reproduce expectation reversal effects due to a connective changing the plausibility of a given scenario; Italian results are even less aligned with human data, with no effects of either plausibility and connectives on Surprisal.</p>","PeriodicalId":12536,"journal":{"name":"Frontiers in Human Neuroscience","volume":"18 ","pages":"1363120"},"PeriodicalIF":2.4000,"publicationDate":"2024-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11471541/pdf/","citationCount":"0","resultStr":"{\"title\":\"On the influence of discourse connectives on the predictions of humans and language models.\",\"authors\":\"James Britton, Yan Cong, Yu-Yin Hsu, Emmanuele Chersoni, Philippe Blache\",\"doi\":\"10.3389/fnhum.2024.1363120\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><p>Psycholinguistic literature has consistently shown that humans rely on a rich and organized understanding of event knowledge to predict the forthcoming linguistic input during online sentence comprehension. We, the authors, expect sentences to maintain coherence with the preceding context, making congruent sentence sequences easier to process than incongruent ones. It is widely known that discourse relations between sentences (e.g., temporal, contingency, comparison) are generally made explicit through specific particles, known as <i>discourse connectives</i>, (e.g., <i>and, but, because, after</i>). However, some relations that are easily accessible to the speakers, given their event knowledge, can also be left implicit. The goal of this paper is to investigate the importance of discourse connectives in the prediction of events in human language processing and pretrained language models, with a specific focus on concessives and contrastives, which signal to comprehenders that their event-related predictions have to be <i>reversed</i>. Inspired by previous work, we built a comprehensive set of story stimuli in Italian and Mandarin Chinese that differ in the plausibility and coherence of the situation being described and the presence or absence of a discourse connective. We collected plausibility judgments and reading times from native speakers for the stimuli. Moreover, we correlated the results of the experiments with the predictions given by computational modeling, using Surprisal scores obtained via Transformer-based language models. The human judgements were collected using a seven-point Likert scale and analyzed using cumulative link mixed modeling (CLMM), while the human reading times and language model surprisal scores were analyzed using linear mixed effects regression (LMER). We found that Chinese NLMs are sensitive to plausibility and connectives, although they struggle to reproduce expectation reversal effects due to a connective changing the plausibility of a given scenario; Italian results are even less aligned with human data, with no effects of either plausibility and connectives on Surprisal.</p>\",\"PeriodicalId\":12536,\"journal\":{\"name\":\"Frontiers in Human Neuroscience\",\"volume\":\"18 \",\"pages\":\"1363120\"},\"PeriodicalIF\":2.4000,\"publicationDate\":\"2024-09-30\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11471541/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Frontiers in Human Neuroscience\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.3389/fnhum.2024.1363120\",\"RegionNum\":3,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"2024/1/1 0:00:00\",\"PubModel\":\"eCollection\",\"JCR\":\"Q3\",\"JCRName\":\"NEUROSCIENCES\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Frontiers in Human Neuroscience","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.3389/fnhum.2024.1363120","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/1/1 0:00:00","PubModel":"eCollection","JCR":"Q3","JCRName":"NEUROSCIENCES","Score":null,"Total":0}
On the influence of discourse connectives on the predictions of humans and language models.
Psycholinguistic literature has consistently shown that humans rely on a rich and organized understanding of event knowledge to predict the forthcoming linguistic input during online sentence comprehension. We, the authors, expect sentences to maintain coherence with the preceding context, making congruent sentence sequences easier to process than incongruent ones. It is widely known that discourse relations between sentences (e.g., temporal, contingency, comparison) are generally made explicit through specific particles, known as discourse connectives, (e.g., and, but, because, after). However, some relations that are easily accessible to the speakers, given their event knowledge, can also be left implicit. The goal of this paper is to investigate the importance of discourse connectives in the prediction of events in human language processing and pretrained language models, with a specific focus on concessives and contrastives, which signal to comprehenders that their event-related predictions have to be reversed. Inspired by previous work, we built a comprehensive set of story stimuli in Italian and Mandarin Chinese that differ in the plausibility and coherence of the situation being described and the presence or absence of a discourse connective. We collected plausibility judgments and reading times from native speakers for the stimuli. Moreover, we correlated the results of the experiments with the predictions given by computational modeling, using Surprisal scores obtained via Transformer-based language models. The human judgements were collected using a seven-point Likert scale and analyzed using cumulative link mixed modeling (CLMM), while the human reading times and language model surprisal scores were analyzed using linear mixed effects regression (LMER). We found that Chinese NLMs are sensitive to plausibility and connectives, although they struggle to reproduce expectation reversal effects due to a connective changing the plausibility of a given scenario; Italian results are even less aligned with human data, with no effects of either plausibility and connectives on Surprisal.
期刊介绍:
Frontiers in Human Neuroscience is a first-tier electronic journal devoted to understanding the brain mechanisms supporting cognitive and social behavior in humans, and how these mechanisms might be altered in disease states. The last 25 years have seen an explosive growth in both the methods and the theoretical constructs available to study the human brain. Advances in electrophysiological, neuroimaging, neuropsychological, psychophysical, neuropharmacological and computational approaches have provided key insights into the mechanisms of a broad range of human behaviors in both health and disease. Work in human neuroscience ranges from the cognitive domain, including areas such as memory, attention, language and perception to the social domain, with this last subject addressing topics, such as interpersonal interactions, social discourse and emotional regulation. How these processes unfold during development, mature in adulthood and often decline in aging, and how they are altered in a host of developmental, neurological and psychiatric disorders, has become increasingly amenable to human neuroscience research approaches. Work in human neuroscience has influenced many areas of inquiry ranging from social and cognitive psychology to economics, law and public policy. Accordingly, our journal will provide a forum for human research spanning all areas of human cognitive, social, developmental and translational neuroscience using any research approach.