The Fill-Mask Association Test (FMAT): Measuring propositions in natural language.

IF 6.4 1区 心理学 Q1 PSYCHOLOGY, SOCIAL Journal of personality and social psychology Pub Date : 2024-09-01 Epub Date: 2024-07-08 DOI:10.1037/pspa0000396
Han-Wu-Shuang Bao
{"title":"The Fill-Mask Association Test (FMAT): Measuring propositions in natural language.","authors":"Han-Wu-Shuang Bao","doi":"10.1037/pspa0000396","DOIUrl":null,"url":null,"abstract":"<p><p>Recent advances in large language models are enabling the computational intelligent analysis of psychology in natural language. Here, the Fill-Mask Association Test (FMAT) is introduced as a novel and integrative method leveraging Masked Language Models to study and measure psychology from a <i>propositional</i> perspective at the societal level. The FMAT uses Bidirectional Encoder Representations from Transformers (BERT) models to compute semantic probabilities of option words filling in the masked blank of a designed query (i.e., a clozelike contextualized sentence). The current research presents 15 studies that establish the reliability and validity of the FMAT in predicting factual associations (Studies 1A-1C), measuring attitudes/biases (Studies 2A-2D), capturing social stereotypes (Studies 3A-3D), and retrospectively delineating lay perceptions of sociocultural changes over time (Studies 4A-4D). Empirically, the FMAT replicated seminal findings previously obtained with human participants (e.g., the Implicit Association Test) and other big-data text-analytic methods (e.g., word frequency analysis, the Word Embedding Association Test), demonstrating robustness across 12 BERT model variants and diverse training text corpora. Theoretically, the current findings substantiate the propositional (vs. associative) perspective on how semantic associations are represented in natural language. Methodologically, the FMAT allows for more fine-grained language-based psychological measurement, with an R package developed to streamline its workflow for use on broader research questions. (PsycInfo Database Record (c) 2024 APA, all rights reserved).</p>","PeriodicalId":16691,"journal":{"name":"Journal of personality and social psychology","volume":" ","pages":"537-561"},"PeriodicalIF":6.4000,"publicationDate":"2024-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of personality and social psychology","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.1037/pspa0000396","RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/7/8 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"PSYCHOLOGY, SOCIAL","Score":null,"Total":0}
引用次数: 0

Abstract

Recent advances in large language models are enabling the computational intelligent analysis of psychology in natural language. Here, the Fill-Mask Association Test (FMAT) is introduced as a novel and integrative method leveraging Masked Language Models to study and measure psychology from a propositional perspective at the societal level. The FMAT uses Bidirectional Encoder Representations from Transformers (BERT) models to compute semantic probabilities of option words filling in the masked blank of a designed query (i.e., a clozelike contextualized sentence). The current research presents 15 studies that establish the reliability and validity of the FMAT in predicting factual associations (Studies 1A-1C), measuring attitudes/biases (Studies 2A-2D), capturing social stereotypes (Studies 3A-3D), and retrospectively delineating lay perceptions of sociocultural changes over time (Studies 4A-4D). Empirically, the FMAT replicated seminal findings previously obtained with human participants (e.g., the Implicit Association Test) and other big-data text-analytic methods (e.g., word frequency analysis, the Word Embedding Association Test), demonstrating robustness across 12 BERT model variants and diverse training text corpora. Theoretically, the current findings substantiate the propositional (vs. associative) perspective on how semantic associations are represented in natural language. Methodologically, the FMAT allows for more fine-grained language-based psychological measurement, with an R package developed to streamline its workflow for use on broader research questions. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
填充-掩码联想测试(FMAT):测量自然语言中的命题。
大型语言模型的最新进展使得对自然语言中的心理学进行计算智能分析成为可能。这里介绍的填充-掩码关联测试(FMAT)是一种利用掩码语言模型从命题角度在社会层面研究和测量心理学的新颖综合方法。FMAT 使用来自变换器的双向编码器表征(BERT)模型来计算选项词填入所设计的查询(即一个掐头去尾的语境化句子)的屏蔽空白的语义概率。目前的研究介绍了 15 项研究,这些研究证实了 FMAT 在预测事实联想(研究 1A-1C)、测量态度/偏见(研究 2A-2D)、捕捉社会刻板印象(研究 3A-3D)以及回溯描述非专业人士对社会文化随时间变化的看法(研究 4A-4D)方面的可靠性和有效性。从经验上讲,FMAT 复制了之前通过人类参与者(如隐性关联测试)和其他大数据文本分析方法(如词频分析、词嵌入关联测试)获得的开创性发现,在 12 个 BERT 模型变体和不同的训练文本库中表现出稳健性。从理论上讲,目前的研究结果证实了命题(与关联)的观点,即语义关联是如何在自然语言中表现出来的。在方法论上,FMAT 允许进行更精细的基于语言的心理测量,并开发了一个 R 软件包来简化其工作流程,以用于更广泛的研究问题。(PsycInfo 数据库记录 (c) 2024 APA,保留所有权利)。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
CiteScore
12.70
自引率
3.90%
发文量
250
期刊介绍: Journal of personality and social psychology publishes original papers in all areas of personality and social psychology and emphasizes empirical reports, but may include specialized theoretical, methodological, and review papers.Journal of personality and social psychology is divided into three independently edited sections. Attitudes and Social Cognition addresses all aspects of psychology (e.g., attitudes, cognition, emotion, motivation) that take place in significant micro- and macrolevel social contexts.
期刊最新文献
Evaluating the psychological and social nature of actual and perceived liking gaps. Reactions to undesired outcomes: Evidence for the opposer's loss effect. Narcissists' experience of ostracism. Stress reactivity and sociocultural learning: More stress-reactive individuals are quicker at learning sociocultural norms from experiential feedback. Money matters for future well-being: A latent growth analysis and meta-analytic integration of associations between income, financial satisfaction, and 22 well-being variables across three data sets.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1