基于模型的共享语言空间,在自然对话中将我们的思想从大脑传递到大脑。

IF 14.7 1区 医学 Q1 NEUROSCIENCES Neuron Pub Date : 2024-09-25 Epub Date: 2024-08-02 DOI:10.1016/j.neuron.2024.06.025
Zaid Zada, Ariel Goldstein, Sebastian Michelmann, Erez Simony, Amy Price, Liat Hasenfratz, Emily Barham, Asieh Zadbood, Werner Doyle, Daniel Friedman, Patricia Dugan, Lucia Melloni, Sasha Devore, Adeen Flinker, Orrin Devinsky, Samuel A Nastase, Uri Hasson
{"title":"基于模型的共享语言空间,在自然对话中将我们的思想从大脑传递到大脑。","authors":"Zaid Zada, Ariel Goldstein, Sebastian Michelmann, Erez Simony, Amy Price, Liat Hasenfratz, Emily Barham, Asieh Zadbood, Werner Doyle, Daniel Friedman, Patricia Dugan, Lucia Melloni, Sasha Devore, Adeen Flinker, Orrin Devinsky, Samuel A Nastase, Uri Hasson","doi":"10.1016/j.neuron.2024.06.025","DOIUrl":null,"url":null,"abstract":"<p><p>Effective communication hinges on a mutual understanding of word meaning in different contexts. We recorded brain activity using electrocorticography during spontaneous, face-to-face conversations in five pairs of epilepsy patients. We developed a model-based coupling framework that aligns brain activity in both speaker and listener to a shared embedding space from a large language model (LLM). The context-sensitive LLM embeddings allow us to track the exchange of linguistic information, word by word, from one brain to another in natural conversations. Linguistic content emerges in the speaker's brain before word articulation and rapidly re-emerges in the listener's brain after word articulation. The contextual embeddings better capture word-by-word neural alignment between speaker and listener than syntactic and articulatory models. Our findings indicate that the contextual embeddings learned by LLMs can serve as an explicit numerical model of the shared, context-rich meaning space humans use to communicate their thoughts to one another.</p>","PeriodicalId":19313,"journal":{"name":"Neuron","volume":" ","pages":"3211-3222.e5"},"PeriodicalIF":14.7000,"publicationDate":"2024-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11427153/pdf/","citationCount":"0","resultStr":"{\"title\":\"A shared model-based linguistic space for transmitting our thoughts from brain to brain in natural conversations.\",\"authors\":\"Zaid Zada, Ariel Goldstein, Sebastian Michelmann, Erez Simony, Amy Price, Liat Hasenfratz, Emily Barham, Asieh Zadbood, Werner Doyle, Daniel Friedman, Patricia Dugan, Lucia Melloni, Sasha Devore, Adeen Flinker, Orrin Devinsky, Samuel A Nastase, Uri Hasson\",\"doi\":\"10.1016/j.neuron.2024.06.025\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><p>Effective communication hinges on a mutual understanding of word meaning in different contexts. We recorded brain activity using electrocorticography during spontaneous, face-to-face conversations in five pairs of epilepsy patients. We developed a model-based coupling framework that aligns brain activity in both speaker and listener to a shared embedding space from a large language model (LLM). The context-sensitive LLM embeddings allow us to track the exchange of linguistic information, word by word, from one brain to another in natural conversations. Linguistic content emerges in the speaker's brain before word articulation and rapidly re-emerges in the listener's brain after word articulation. The contextual embeddings better capture word-by-word neural alignment between speaker and listener than syntactic and articulatory models. Our findings indicate that the contextual embeddings learned by LLMs can serve as an explicit numerical model of the shared, context-rich meaning space humans use to communicate their thoughts to one another.</p>\",\"PeriodicalId\":19313,\"journal\":{\"name\":\"Neuron\",\"volume\":\" \",\"pages\":\"3211-3222.e5\"},\"PeriodicalIF\":14.7000,\"publicationDate\":\"2024-09-25\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11427153/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Neuron\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1016/j.neuron.2024.06.025\",\"RegionNum\":1,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"2024/8/2 0:00:00\",\"PubModel\":\"Epub\",\"JCR\":\"Q1\",\"JCRName\":\"NEUROSCIENCES\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Neuron","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1016/j.neuron.2024.06.025","RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/8/2 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"NEUROSCIENCES","Score":null,"Total":0}
引用次数: 0

摘要

有效的交流取决于对不同语境中词汇含义的相互理解。我们使用皮层电图记录了五对癫痫患者自发面对面交谈时的大脑活动。我们开发了一个基于模型的耦合框架,将说话者和听话者的大脑活动与大语言模型(LLM)的共享嵌入空间相匹配。通过对上下文敏感的 LLM 嵌入,我们可以跟踪自然对话中一个大脑与另一个大脑逐字交换语言信息的情况。语言内容在单词发音前出现在说话者的大脑中,并在单词发音后迅速重新出现在听者的大脑中。与句法和发音模型相比,语境嵌入能更好地捕捉说话者和听话者之间的逐字神经一致性。我们的研究结果表明,由 LLMs 学习到的语境嵌入可以作为一个明确的数字模型,用于描述人类用来相互交流思想的共享的、语境丰富的意义空间。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
A shared model-based linguistic space for transmitting our thoughts from brain to brain in natural conversations.

Effective communication hinges on a mutual understanding of word meaning in different contexts. We recorded brain activity using electrocorticography during spontaneous, face-to-face conversations in five pairs of epilepsy patients. We developed a model-based coupling framework that aligns brain activity in both speaker and listener to a shared embedding space from a large language model (LLM). The context-sensitive LLM embeddings allow us to track the exchange of linguistic information, word by word, from one brain to another in natural conversations. Linguistic content emerges in the speaker's brain before word articulation and rapidly re-emerges in the listener's brain after word articulation. The contextual embeddings better capture word-by-word neural alignment between speaker and listener than syntactic and articulatory models. Our findings indicate that the contextual embeddings learned by LLMs can serve as an explicit numerical model of the shared, context-rich meaning space humans use to communicate their thoughts to one another.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Neuron
Neuron 医学-神经科学
CiteScore
24.50
自引率
3.10%
发文量
382
审稿时长
1 months
期刊介绍: Established as a highly influential journal in neuroscience, Neuron is widely relied upon in the field. The editors adopt interdisciplinary strategies, integrating biophysical, cellular, developmental, and molecular approaches alongside a systems approach to sensory, motor, and higher-order cognitive functions. Serving as a premier intellectual forum, Neuron holds a prominent position in the entire neuroscience community.
期刊最新文献
Meningeal neutrophil immune signaling influences behavioral adaptation following threat. Stability of cross-sensory input to primary somatosensory cortex across experience. Appoptosin-Mediated Caspase Cleavage of Tau Contributes to Progressive Supranuclear Palsy Pathogenesis. Network-wide risk convergence in gene co-expression identifies reproducible genetic hubs of schizophrenia risk. Failure in a population: Tauopathy disrupts homeostatic set-points in emergent dynamics despite stability in the constituent neurons.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1