首页 > 最新文献

Social Psychological Bulletin最新文献

英文 中文
On the importance of modeling the invisible world of underlying effect sizes 对基本效应大小的隐形世界进行建模的重要性
Q2 Psychology Pub Date : 2023-11-17 DOI: 10.32872/spb.9981
Brent M Wilson, J. Wixted
The headline findings from the Open Science Collaboration (2015)―namely, that 36% of original experiments replicated at p < .05, with the overall replication effect sizes being half as large as the original effects―cannot be meaningfully interpreted without a formal model. A simple model-based approach might ask: what would the state of original science be and what would replication results show if original experiments tested true effects half the time (prior odds = 1), true effects had a medium effect size (Cohen’s δ = 0.50), and power to detect true effects was 50%? Assuming no questionable research practices, 91% of p < .05 findings in the original literature would be true positives. However, only 58% of original p < .05 findings would be expected to replicate using the Open Science Collaboration approach, and the replication effects overall would be only ~60% as large as the original effects. A minor variant of this model yields an expected replication rate of only 45%, with overall replication effect sizes dropping by half. If the state of original science is as grim as a non-model-based (i.e., intuitive) interpretation of the Open Science Collaboration data suggests, should it be this easy to largely account for those findings using a model in which 91% of statistically significant findings in the original science literature are true positives? Claims that the findings reported by the Open Science Collaboration indicate a replication crisis should not be based solely on intuition but should instead be accompanied by a specific model that supports that interpretation.
开放科学合作组织(2015)的主要发现--即 36% 的原始实验的复制结果 p < .05,总体复制效应大小是原始效应的一半--如果没有一个正式的模型,是无法进行有意义的解释的。一个简单的基于模型的方法可能会问:如果原始实验有一半的时间测试了真实效应(先验几率 = 1),真实效应具有中等效应大小(Cohen's δ = 0.50),检测真实效应的功率为 50%,那么原始科学的状况会是怎样的?假设没有可疑的研究实践,那么原始文献中 91% 的 p < .05 结果将是真正的阳性结果。然而,使用开放科学合作方法,预计只有 58% 的 p < .05 的原始研究结果会被复制,复制效果总体上只有原始效果的 60%。该模型的一个小变体得出的预期复制率仅为 45%,总体复制效应大小下降了一半。如果原创科学的现状就像对开放科学合作组织数据的非模型(即直观)解释所显示的那样严峻,那么使用一个模型来解释这些发现是否就这么容易呢?在这个模型中,原创科学文献中91%具有统计意义的发现都是真阳性?关于开放科学合作组织报告的研究结果表明存在复制危机的说法不应该仅仅基于直觉,而应该有一个支持这种解释的具体模型。
{"title":"On the importance of modeling the invisible world of underlying effect sizes","authors":"Brent M Wilson, J. Wixted","doi":"10.32872/spb.9981","DOIUrl":"https://doi.org/10.32872/spb.9981","url":null,"abstract":"The headline findings from the Open Science Collaboration (2015)―namely, that 36% of original experiments replicated at p < .05, with the overall replication effect sizes being half as large as the original effects―cannot be meaningfully interpreted without a formal model. A simple model-based approach might ask: what would the state of original science be and what would replication results show if original experiments tested true effects half the time (prior odds = 1), true effects had a medium effect size (Cohen’s δ = 0.50), and power to detect true effects was 50%? Assuming no questionable research practices, 91% of p < .05 findings in the original literature would be true positives. However, only 58% of original p < .05 findings would be expected to replicate using the Open Science Collaboration approach, and the replication effects overall would be only ~60% as large as the original effects. A minor variant of this model yields an expected replication rate of only 45%, with overall replication effect sizes dropping by half. If the state of original science is as grim as a non-model-based (i.e., intuitive) interpretation of the Open Science Collaboration data suggests, should it be this easy to largely account for those findings using a model in which 91% of statistically significant findings in the original science literature are true positives? Claims that the findings reported by the Open Science Collaboration indicate a replication crisis should not be based solely on intuition but should instead be accompanied by a specific model that supports that interpretation.","PeriodicalId":32922,"journal":{"name":"Social Psychological Bulletin","volume":"12 4","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139263677","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Evading open science: The black box of student data collection 逃避开放科学:学生数据收集的黑匣子
Q2 Psychology Pub Date : 2023-11-17 DOI: 10.32872/spb.9411
Tobias Ludwig, M. S. Altenmüller, Leonhard Falk Florentin Schramm, Mathias Twardawski
While Open Science has arguably initiated positive changes at some stages of the research process (e.g., increasing transparency through preregistration), problematic behaviors during data collection are still almost impossible to detect and pose a great risk to the validity and integrity of psychological research—especially, when researchers use data collected by others (e.g., students). Exploring students’ and supervisors’ perspectives, the present registered report enlightens this “black box” of student data collection, focusing on questionable research practices and research misconduct (QRP/M). The majority of students did not report having engaged in any problematic behaviors during data collection, but some QRP/M—ranging from somewhat questionable to highly fraudulent—seem quite common (e.g., telling participants the hypothesis beforehand, participating in one’s own survey). We provide an overview of students’ reported and supervisors’ suspected data collection QRP/M, explore potential drivers for these behaviors based on the fraud triangle model (including pressures, opportunities, and rationalizations), and report how students and supervisors perceive the eligibility of student data for further uses (e.g., scientific publications). Moreover, we explore the role of the student-supervisor relationship (e.g., communication and expectations) and Open Science practices in student projects. In summary, our findings suggest the potential scientific value of data from student projects. Fostering transparent communication regarding expectations, experiences, and intentions between supervisors and students might further contribute to strengthening this prospect.
虽然开放科学在研究过程的某些阶段(如通过预注册提高透明度)可以说已经引发了积极的变化,但数据收集过程中的问题行为仍然几乎不可能被发现,并对心理学研究的有效性和完整性构成了极大的风险--尤其是当研究人员使用他人(如学生)收集的数据时。本注册报告从学生和督导的角度出发,揭示了学生数据收集的 "黑匣子",重点关注有问题的研究实践和研究不当行为(QRP/M)。大多数学生没有报告在数据收集过程中出现过任何问题行为,但一些QRP/M--从有点问题到高度欺诈--似乎相当普遍(例如,事先告诉参与者假设,参与自己的调查)。我们概述了学生报告和督导怀疑的数据收集 QRP/M,根据欺诈三角模型(包括压力、机会和合理化)探讨了这些行为的潜在驱动因素,并报告了学生和督导如何看待学生数据的进一步使用资格(如科学出版物)。此外,我们还探讨了学生与导师之间的关系(如沟通和期望)以及开放科学实践在学生项目中的作用。总之,我们的研究结果表明,学生项目的数据具有潜在的科学价值。促进导师与学生之间在期望、经验和意向方面的透明沟通,可能会进一步有助于加强这一前景。
{"title":"Evading open science: The black box of student data collection","authors":"Tobias Ludwig, M. S. Altenmüller, Leonhard Falk Florentin Schramm, Mathias Twardawski","doi":"10.32872/spb.9411","DOIUrl":"https://doi.org/10.32872/spb.9411","url":null,"abstract":"While Open Science has arguably initiated positive changes at some stages of the research process (e.g., increasing transparency through preregistration), problematic behaviors during data collection are still almost impossible to detect and pose a great risk to the validity and integrity of psychological research—especially, when researchers use data collected by others (e.g., students). Exploring students’ and supervisors’ perspectives, the present registered report enlightens this “black box” of student data collection, focusing on questionable research practices and research misconduct (QRP/M). The majority of students did not report having engaged in any problematic behaviors during data collection, but some QRP/M—ranging from somewhat questionable to highly fraudulent—seem quite common (e.g., telling participants the hypothesis beforehand, participating in one’s own survey). We provide an overview of students’ reported and supervisors’ suspected data collection QRP/M, explore potential drivers for these behaviors based on the fraud triangle model (including pressures, opportunities, and rationalizations), and report how students and supervisors perceive the eligibility of student data for further uses (e.g., scientific publications). Moreover, we explore the role of the student-supervisor relationship (e.g., communication and expectations) and Open Science practices in student projects. In summary, our findings suggest the potential scientific value of data from student projects. Fostering transparent communication regarding expectations, experiences, and intentions between supervisors and students might further contribute to strengthening this prospect.","PeriodicalId":32922,"journal":{"name":"Social Psychological Bulletin","volume":"38 6","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139265739","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Introduction to special topic “Is psychology self-correcting? Reflections on the credibility revolution in social and personality psychology” 特别专题 "心理学是自我矫正的吗?对社会和人格心理学公信力革命的思考"
Q2 Psychology Pub Date : 2023-11-17 DOI: 10.32872/spb.12927
S. Vazire, Brian Nosek
{"title":"Introduction to special topic “Is psychology self-correcting? Reflections on the credibility revolution in social and personality psychology”","authors":"S. Vazire, Brian Nosek","doi":"10.32872/spb.12927","DOIUrl":"https://doi.org/10.32872/spb.12927","url":null,"abstract":"","PeriodicalId":32922,"journal":{"name":"Social Psychological Bulletin","volume":"81 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139263293","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
The “replication crisis” and trust in psychological science: How reforms shape public trust in psychology 复制危机 "与对心理科学的信任:改革如何塑造公众对心理学的信任
Q2 Psychology Pub Date : 2023-11-17 DOI: 10.32872/spb.9665
Nicole Methner, Barbara Dahme, Claudia Menzel
Failed replications can jeopardize public trust in psychological science and recent findings cast doubt on the idea that self-corrections and reforms can rebuild this trust. These findings are in contrast to trust repair research that proposes changes in transparency, norms, and policies as trust repair mechanisms. This raises the question of whether the used experimental material is one reason behind these unexpected findings. Previous studies used short texts that may give too little information on the replication crisis and initiated reforms in the field. In a pre-registered experiment (N = 390), we, therefore, tested whether comprehensive information about the replication crisis and reforms increases public trust in psychology, compared to a control condition that only informs about the replication crisis. To give comprehensive information, we created an animated video for each experimental condition. After watching the video, participants indicated their trust in researchers, trust in past research findings, and trust in current research findings. As expected and in line with trust repair research, information about reforms increased trust in researchers and in current (vs. past) research, compared with information about the replication crisis and its causes only. We discuss the generalizability of our results and implications for communicating the replication crisis to the public.
失败的复制会危及公众对心理科学的信任,而最近的研究结果让人对自我纠正和改革可以重建这种信任的想法产生了怀疑。这些研究结果与信任修复研究形成了鲜明对比,后者建议将透明度、规范和政策的改变作为信任修复机制。这就提出了一个问题,即所使用的实验材料是否是这些意外发现背后的原因之一。以前的研究使用的是短文,可能对复制危机和该领域启动的改革提供的信息太少。因此,在一项预先登记的实验(N = 390)中,我们测试了与只告知复制危机的对照条件相比,有关复制危机和改革的全面信息是否会增加公众对心理学的信任。为了提供全面的信息,我们为每个实验条件制作了一段动画视频。在观看视频后,参与者分别表达了他们对研究人员的信任、对过去研究成果的信任以及对当前研究成果的信任。正如预期的那样,与信任修复研究相一致的是,与只提供复制危机及其原因的信息相比,有关改革的信息增加了对研究人员和当前(相对于过去)研究的信任。我们将讨论结果的普遍性以及向公众宣传复制危机的意义。
{"title":"The “replication crisis” and trust in psychological science: How reforms shape public trust in psychology","authors":"Nicole Methner, Barbara Dahme, Claudia Menzel","doi":"10.32872/spb.9665","DOIUrl":"https://doi.org/10.32872/spb.9665","url":null,"abstract":"Failed replications can jeopardize public trust in psychological science and recent findings cast doubt on the idea that self-corrections and reforms can rebuild this trust. These findings are in contrast to trust repair research that proposes changes in transparency, norms, and policies as trust repair mechanisms. This raises the question of whether the used experimental material is one reason behind these unexpected findings. Previous studies used short texts that may give too little information on the replication crisis and initiated reforms in the field. In a pre-registered experiment (N = 390), we, therefore, tested whether comprehensive information about the replication crisis and reforms increases public trust in psychology, compared to a control condition that only informs about the replication crisis. To give comprehensive information, we created an animated video for each experimental condition. After watching the video, participants indicated their trust in researchers, trust in past research findings, and trust in current research findings. As expected and in line with trust repair research, information about reforms increased trust in researchers and in current (vs. past) research, compared with information about the replication crisis and its causes only. We discuss the generalizability of our results and implications for communicating the replication crisis to the public.","PeriodicalId":32922,"journal":{"name":"Social Psychological Bulletin","volume":"10 1-2","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139263828","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Hits and misses in the last decade of open science: Researchers from different subfields and career stages offer personal reflections and suggestions 过去十年开放科学的成功与失败:来自不同子领域和职业阶段的研究人员提出个人反思和建议
Q2 Psychology Pub Date : 2023-11-17 DOI: 10.32872/spb.9681
Jennifer Beer, Paul Eastwick, Jin X. Goh
The success of Open Science in addressing the replication crisis and restoring credibility in psychology can be understood more completely by examining the successes and challenges of adapting the recommended best practices by researchers at different types of institutions, different career stages, and from different subfields within social and personality psychology. In this article, we offer personal reflections about the impact and future of the Open Science movement in a conversational form between three researchers at varying career stages who focus on different subdisciplines (relationship science, diversity science, and social neuroscience and social cognition) and work at universities that place a different emphasis on research (relative to teaching and service). We see many successes of the open science movement, but we also note that implementation has trailed behind its full potential because (a) the incentive structures of our existing rigid system remain misaligned with some open-science goals, and (b) some open science solutions were designed by researchers with certain types of scientific practices in mind. We all feel encouraged by the focus on larger samples, greater data sharing, and pre-registration both for experimental design and analytical decisions. However, there are areas that need attention. Our perspective is that the open science movement has not been as strong of an ally as it could be to another goal of psychological science: increasing diversity, equity, and inclusion. Additionally, more careful consideration of how to facilitate data sharing and pre-registration is needed and may necessitate a shift in the incentive structure of our field.
通过研究不同类型机构、不同职业阶段以及来自社会与人格心理学不同子领域的研究人员在采用所推荐的最佳实践时取得的成功与面临的挑战,我们可以更全面地了解开放科学在解决复制危机和恢复心理学可信度方面所取得的成功。在这篇文章中,我们以对话的形式,让三位处于不同职业阶段、专注于不同分支学科(关系科学、多样性科学、社会神经科学与社会认知)、在不同强调研究(相对于教学和服务)的大学工作的研究人员,就开放科学运动的影响和未来进行个人反思。我们看到了开放科学运动的许多成功之处,但我们也注意到,其实施落后于其全部潜力,这是因为:(a)我们现有的僵化系统的激励结构仍然与某些开放科学目标不一致,以及(b)某些开放科学解决方案是由研究人员根据某些类型的科学实践而设计的。我们都感到鼓舞的是,对更大样本、更多数据共享以及实验设计和分析决策预注册的重视。然而,还有一些领域需要关注。我们认为,开放科学运动在实现心理科学的另一个目标--提高多样性、公平性和包容性--方面并没有起到应有的作用。此外,我们还需要更仔细地考虑如何促进数据共享和预注册,这可能需要改变我们领域的激励结构。
{"title":"Hits and misses in the last decade of open science: Researchers from different subfields and career stages offer personal reflections and suggestions","authors":"Jennifer Beer, Paul Eastwick, Jin X. Goh","doi":"10.32872/spb.9681","DOIUrl":"https://doi.org/10.32872/spb.9681","url":null,"abstract":"The success of Open Science in addressing the replication crisis and restoring credibility in psychology can be understood more completely by examining the successes and challenges of adapting the recommended best practices by researchers at different types of institutions, different career stages, and from different subfields within social and personality psychology. In this article, we offer personal reflections about the impact and future of the Open Science movement in a conversational form between three researchers at varying career stages who focus on different subdisciplines (relationship science, diversity science, and social neuroscience and social cognition) and work at universities that place a different emphasis on research (relative to teaching and service). We see many successes of the open science movement, but we also note that implementation has trailed behind its full potential because (a) the incentive structures of our existing rigid system remain misaligned with some open-science goals, and (b) some open science solutions were designed by researchers with certain types of scientific practices in mind. We all feel encouraged by the focus on larger samples, greater data sharing, and pre-registration both for experimental design and analytical decisions. However, there are areas that need attention. Our perspective is that the open science movement has not been as strong of an ally as it could be to another goal of psychological science: increasing diversity, equity, and inclusion. Additionally, more careful consideration of how to facilitate data sharing and pre-registration is needed and may necessitate a shift in the incentive structure of our field.","PeriodicalId":32922,"journal":{"name":"Social Psychological Bulletin","volume":"31 2","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139264344","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Psychology’s reform movement needs a reconceptualization of scientific expertise 心理学改革运动需要重新认识科学专业知识
Q2 Psychology Pub Date : 2023-11-17 DOI: 10.32872/spb.10303
Duygu Uygun Tunç, M. Tunç
Science is supposed to be a self-correcting endeavor, but who is “the scientific expert” that corrects faulty science? We grouped traditional conceptualizations of expertise in psychology under three classes (substantialist, implicitist, and social conventionalist), and then examined how these approaches affect scientific self-correction in reference to various components of the credibility crisis such as fraud/QRPs, the inadequate number of replication studies, challenges facing big team science, and perverse incentives. Our investigation pointed out several problems with the traditional views. First, traditional views conceptualize expertise as something possessed, not performed, ignoring the epistemic responsibility of experts. Second, expertise is conceived as an exclusively individual quality, which contradicts the socially distributed nature of scientific inquiry. Third, some aspects of expertise are taken to be implicit or relative to the established research practices in a field, which leads to disputes over replicability and makes it difficult to criticize mindless scientific rituals. Lastly, a conflation of expertise with eminence in practice creates an incentive structure that undermines the goal of self-correction in science. We suggest, instead, that we conceive an expert as a reliable informant. Following the extended virtue account of expertise, we propose a non-individualist and a performance-based model, and discuss why it does not suffer from the same problems as traditional approaches, and why it is more compatible with the reform movement's goal of creating a credible psychological science through self-correction.
科学应该是一种自我纠正的努力,但谁是纠正错误科学的 "科学专家 "呢?我们将心理学中关于专业知识的传统概念分为三类(实质主义、隐含主义和社会传统主义),然后参照可信度危机的各个组成部分,如欺诈/质量保证计划、复制研究数量不足、大团队科学面临的挑战以及不正当激励机制,研究了这些方法如何影响科学的自我纠正。我们的调查指出了传统观点的几个问题。首先,传统观点将专业知识概念化为一种拥有而非执行的东西,忽视了专家的认识论责任。其次,专业知识被视为一种纯粹的个人素质,这与科学探索的社会分布性质相矛盾。第三,专业知识的某些方面被认为是隐含的或相对于某一领域的既定研究实践而言的,这导致了关于可复制性的争议,并使人们难以批评无谓的科学仪式。最后,将专业知识与实践中的杰出表现混为一谈会产生一种激励结构,从而破坏科学的自我修正目标。我们建议将专家视为可靠的信息提供者。根据对专业知识的扩展美德论述,我们提出了一个非个人主义和基于绩效的模型,并讨论了为什么它不会出现与传统方法相同的问题,以及为什么它更符合改革运动的目标,即通过自我纠正创建可信的心理科学。
{"title":"Psychology’s reform movement needs a reconceptualization of scientific expertise","authors":"Duygu Uygun Tunç, M. Tunç","doi":"10.32872/spb.10303","DOIUrl":"https://doi.org/10.32872/spb.10303","url":null,"abstract":"Science is supposed to be a self-correcting endeavor, but who is “the scientific expert” that corrects faulty science? We grouped traditional conceptualizations of expertise in psychology under three classes (substantialist, implicitist, and social conventionalist), and then examined how these approaches affect scientific self-correction in reference to various components of the credibility crisis such as fraud/QRPs, the inadequate number of replication studies, challenges facing big team science, and perverse incentives. Our investigation pointed out several problems with the traditional views. First, traditional views conceptualize expertise as something possessed, not performed, ignoring the epistemic responsibility of experts. Second, expertise is conceived as an exclusively individual quality, which contradicts the socially distributed nature of scientific inquiry. Third, some aspects of expertise are taken to be implicit or relative to the established research practices in a field, which leads to disputes over replicability and makes it difficult to criticize mindless scientific rituals. Lastly, a conflation of expertise with eminence in practice creates an incentive structure that undermines the goal of self-correction in science. We suggest, instead, that we conceive an expert as a reliable informant. Following the extended virtue account of expertise, we propose a non-individualist and a performance-based model, and discuss why it does not suffer from the same problems as traditional approaches, and why it is more compatible with the reform movement's goal of creating a credible psychological science through self-correction.","PeriodicalId":32922,"journal":{"name":"Social Psychological Bulletin","volume":"2 3","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139264797","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Investigating lay perceptions of psychological measures: A registered report 调查外行人对心理测量的看法:登记报告
Q2 Psychology Pub Date : 2023-11-17 DOI: 10.32872/spb.9383
Joseph Mason, Madeleine Pownall, Amy Palmer, F. Azevedo
In recent years, the reliability and validity of psychology measurement practices has been called into question, as part of an ongoing reappraisal of the robustness, reproducibility, and transparency of psychological research. While useful progress has been made, to date, the majority of discussions surrounding psychology’s measurement crisis have involved technical, quantitative investigations into the validity, reliability, and statistical robustness of psychological measures. This registered report offers a seldom-heard qualitative perspective on these ongoing debates, critically exploring members of the general public’s (i.e., non-experts) lay perceptions of widely used measures in psychology. Using a combination of cognitive interviews and a think aloud study protocol, participants (n = 23) completed one of three popular psychology measures. Participants reflected on each of the measures, discussed the contents, and provided perceptions of what the measures are designed to test. Coding of the think aloud protocols showed that participants across the measures had issues in interpreting and responding to items. Thematic analysis of the cognitive interviews identified three dominant themes that each relate to lay perceptions of psychology measurements. These were: (1) participants’ grappling with attempting to ‘capture their multiple selves’ in the questionnaires, (2) participants perceiving the questionnaire method as generally ‘missing nuance and richness’ and (3) exposing the ‘hidden labour of questionnaires’. These findings are discussed in the context of psychology’s measurement reform.
近年来,心理学测量实践的可靠性和有效性受到了质疑,这也是心理学研究的稳健性、可重复性和透明度的持续重新评估的一部分。虽然已经取得了有益的进展,但迄今为止,围绕心理学测量危机的讨论大多涉及心理学测量的有效性、可靠性和统计稳健性的技术性定量调查。这份注册报告为这些正在进行的争论提供了一个鲜为人知的定性视角,批判性地探讨了普通大众(即非专业人士)对心理学中广泛使用的测量方法的非专业看法。通过认知访谈和大声思考研究协议的结合,参与者(n = 23)完成了三种流行心理学测量方法中的一种。参与者对每项测量进行了反思,讨论了测量内容,并提供了他们对这些测量旨在测试什么的看法。对朗读思考协议的编码显示,不同测量项目的参与者在解释和回应项目时都存在问题。认知访谈的主题分析确定了三个主导主题,每个主题都与外行人对心理学测量的看法有关。它们是(1) 参与者努力尝试在问卷中 "捕捉多重自我",(2) 参与者认为问卷调查方法通常 "缺少细微差别和丰富性",(3) 暴露了 "问卷调查的隐性劳动"。这些研究结果将在心理学测量改革的背景下进行讨论。
{"title":"Investigating lay perceptions of psychological measures: A registered report","authors":"Joseph Mason, Madeleine Pownall, Amy Palmer, F. Azevedo","doi":"10.32872/spb.9383","DOIUrl":"https://doi.org/10.32872/spb.9383","url":null,"abstract":"In recent years, the reliability and validity of psychology measurement practices has been called into question, as part of an ongoing reappraisal of the robustness, reproducibility, and transparency of psychological research. While useful progress has been made, to date, the majority of discussions surrounding psychology’s measurement crisis have involved technical, quantitative investigations into the validity, reliability, and statistical robustness of psychological measures. This registered report offers a seldom-heard qualitative perspective on these ongoing debates, critically exploring members of the general public’s (i.e., non-experts) lay perceptions of widely used measures in psychology. Using a combination of cognitive interviews and a think aloud study protocol, participants (n = 23) completed one of three popular psychology measures. Participants reflected on each of the measures, discussed the contents, and provided perceptions of what the measures are designed to test. Coding of the think aloud protocols showed that participants across the measures had issues in interpreting and responding to items. Thematic analysis of the cognitive interviews identified three dominant themes that each relate to lay perceptions of psychology measurements. These were: (1) participants’ grappling with attempting to ‘capture their multiple selves’ in the questionnaires, (2) participants perceiving the questionnaire method as generally ‘missing nuance and richness’ and (3) exposing the ‘hidden labour of questionnaires’. These findings are discussed in the context of psychology’s measurement reform.","PeriodicalId":32922,"journal":{"name":"Social Psychological Bulletin","volume":"51 2","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139265468","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Strategic ambiguity in the social sciences 社会科学中的战略模糊性
Q2 Psychology Pub Date : 2023-11-17 DOI: 10.32872/spb.9923
W. Frankenhuis, Karthik Panchanathan, P. Smaldino
In the wake of the replication crisis, there have been calls to increase the clarity and precision of theory in the social sciences. Here, we argue that the effects of these calls may be limited due to incentives favoring ambiguous theory. Intentionally or not, scientists can exploit theoretical ambiguities to make support for a claim appear stronger than it is. Practices include theory stretching, interpreting an ambiguous claim more expansively to absorb data outside of the scope of the original claim, and post-hoc precision, interpreting an ambiguous claim more narrowly so it appears more precisely aligned with the data. These practices lead to the overestimation of evidence for the original claim and create the appearance of consistent support and progressive research programs, which may in turn be rewarded by journals, funding agencies, and hiring committees. Selection for ambiguous research can occur even when scientists act in good faith. Although ambiguity might be inevitable or even useful in the early stages of theory construction, scientists should aim for increased clarity as knowledge advances. Science benefits from transparently communicating about known ambiguities. To attain transparency about ambiguity, we provide a set of recommendations for authors, reviewers, and journals. We conclude with suggestions for research on how scientists use strategic ambiguity to advance their careers and the ways in which norms, incentives, and practices favor strategic ambiguity. Our paper ends with a simple mathematical model exploring the conditions in which high-ambiguity theories are favored over low-ambiguity theories, providing a basis for future analyses.
在复制危机之后,人们呼吁提高社会科学理论的清晰度和精确度。在此,我们认为,这些呼吁的效果可能有限,原因在于人们倾向于模糊的理论。无论有意还是无意,科学家们都会利用理论上的模糊性来使某一主张的支持性看起来比实际更强。这种做法包括理论延伸和事后精确,前者是指对模棱两可的主张进行更宽泛的解释,以吸收原始主张范围之外的数据;后者是指对模棱两可的主张进行更狭义的解释,以使其看起来与数据更精确地保持一致。这些做法会导致过高估计原始主张的证据,并造成一贯支持和不断进步的研究计划的表象,进而可能得到期刊、资助机构和招聘委员会的奖励。即使科学家出于善意,也可能会选择模棱两可的研究。尽管在理论构建的早期阶段,模糊性可能是不可避免的,甚至是有用的,但随着知识的进步,科学家应力求更加清晰。以透明的方式交流已知的模糊性对科学有益。为了实现模糊性的透明化,我们为作者、审稿人和期刊提出了一系列建议。最后,我们就科学家如何利用策略性模糊来促进其职业发展,以及规范、激励和实践如何有利于策略性模糊的研究提出了建议。最后,我们用一个简单的数学模型探讨了高模糊性理论比低模糊性理论更受青睐的条件,为未来的分析提供了基础。
{"title":"Strategic ambiguity in the social sciences","authors":"W. Frankenhuis, Karthik Panchanathan, P. Smaldino","doi":"10.32872/spb.9923","DOIUrl":"https://doi.org/10.32872/spb.9923","url":null,"abstract":"In the wake of the replication crisis, there have been calls to increase the clarity and precision of theory in the social sciences. Here, we argue that the effects of these calls may be limited due to incentives favoring ambiguous theory. Intentionally or not, scientists can exploit theoretical ambiguities to make support for a claim appear stronger than it is. Practices include theory stretching, interpreting an ambiguous claim more expansively to absorb data outside of the scope of the original claim, and post-hoc precision, interpreting an ambiguous claim more narrowly so it appears more precisely aligned with the data. These practices lead to the overestimation of evidence for the original claim and create the appearance of consistent support and progressive research programs, which may in turn be rewarded by journals, funding agencies, and hiring committees. Selection for ambiguous research can occur even when scientists act in good faith. Although ambiguity might be inevitable or even useful in the early stages of theory construction, scientists should aim for increased clarity as knowledge advances. Science benefits from transparently communicating about known ambiguities. To attain transparency about ambiguity, we provide a set of recommendations for authors, reviewers, and journals. We conclude with suggestions for research on how scientists use strategic ambiguity to advance their careers and the ways in which norms, incentives, and practices favor strategic ambiguity. Our paper ends with a simple mathematical model exploring the conditions in which high-ambiguity theories are favored over low-ambiguity theories, providing a basis for future analyses.","PeriodicalId":32922,"journal":{"name":"Social Psychological Bulletin","volume":"9 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139263558","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Assessing the reliability of an infrared thermography protocol to assess cold-induced Brown Adipose Tissue activation in french psychology students 评估红外热成像协议的可靠性,以评估冷诱导棕色脂肪组织激活在法国心理学学生
Q2 Psychology Pub Date : 2023-10-12 DOI: 10.32872/spb.10461
Nathan Vidal, Joseph T. Costello, Blandine Ribotta, Lilas Gurgand, Hans IJzerman

The authors use infrared thermography measurements of skin temperature to non-invasively assess the heat production of Brown Adipose Tissue (BAT). In species other than humans, BAT has been linked to maternal care, and may thus be crucial for understanding differences in attachment security. Whereas early BAT research measured its relative presence in the human body through radioactive tracers, researchers have recently used infrared thermography measurement of skin temperature in cold conditions to study BAT thermogenesis outside of medical facilities. Infrared thermography relies on comparing skin temperature in the supraclavicular region (where a BAT depot is located) with skin temperature in the sternal region (which contains no BAT depots) in cold conditions, when the supraclavicular BAT depot produces heat. We replicated an infrared thermography protocol, which previously reported an increase of 0.2 °C in supraclavicular (vs. sternal) skin temperature in cold (vs. control) conditions in only 7 adults, which probably led to overestimation of the effect. With a much larger sample size (N = 94 young adults) and a similar protocol, we did not find any significant variation in relative, Cohen’s d = 0.10, 95% CI [-0.31, 0.50], or absolute supraclavicular skin temperature, Cohen’s d = 0.11, 95% CI [-0.30, 0.52]. Using conditional random forests, we also excluded a variety of alternative explanations for why the method failed to achieve an effect. This protocol of infrared thermography cannot measure BAT thermogenesis and is thus not recommended for future studies to study individual differences in attachment.

作者使用皮肤温度的红外热成像测量来非侵入性评估棕色脂肪组织(BAT)的产热。在人类以外的物种中,BAT与母性护理有关,因此可能对理解依恋安全的差异至关重要。早期的BAT研究通过放射性示踪剂测量其在人体中的相对存在,而研究人员最近使用红外热像仪测量寒冷条件下的皮肤温度来研究医疗设施外BAT的产热作用。红外热成像依赖于比较锁骨上区域(BAT仓库所在地)的皮肤温度与胸骨区域(不包含BAT仓库)在寒冷条件下的皮肤温度,当锁骨上BAT仓库产生热量时。我们复制了一项红外热成像方案,该方案先前仅报道了7名成年人在寒冷条件下锁骨上(相对于胸骨)皮肤温度升高0.2°C,这可能导致了对效果的高估。在更大的样本量(N = 94名年轻人)和类似的方案中,我们没有发现相对、Cohen’s d = 0.10, 95% CI[-0.31, 0.50]或绝对锁骨上皮肤温度的任何显著变化,Cohen’s d = 0.11, 95% CI[-0.30, 0.52]。使用条件随机森林,我们还排除了各种替代解释,为什么该方法未能达到效果。这种红外热成像方案不能测量BAT产热,因此不推荐用于未来研究依恋个体差异的研究。
{"title":"Assessing the reliability of an infrared thermography protocol to assess cold-induced Brown Adipose Tissue activation in french psychology students","authors":"Nathan Vidal, Joseph T. Costello, Blandine Ribotta, Lilas Gurgand, Hans IJzerman","doi":"10.32872/spb.10461","DOIUrl":"https://doi.org/10.32872/spb.10461","url":null,"abstract":"<p xmlns=\"http://www.ncbi.nlm.nih.gov/JATS1\">The authors use infrared thermography measurements of skin temperature to non-invasively assess the heat production of Brown Adipose Tissue (BAT). In species other than humans, BAT has been linked to maternal care, and may thus be crucial for understanding differences in attachment security. Whereas early BAT research measured its relative presence in the human body through radioactive tracers, researchers have recently used infrared thermography measurement of skin temperature in cold conditions to study BAT thermogenesis outside of medical facilities. Infrared thermography relies on comparing skin temperature in the supraclavicular region (where a BAT depot is located) with skin temperature in the sternal region (which contains no BAT depots) in cold conditions, when the supraclavicular BAT depot produces heat. We replicated an infrared thermography protocol, which previously reported an increase of 0.2 °C in supraclavicular (vs. sternal) skin temperature in cold (vs. control) conditions in only 7 adults, which probably led to overestimation of the effect. With a much larger sample size (N = 94 young adults) and a similar protocol, we did not find any significant variation in relative, Cohen’s d = 0.10, 95% CI [-0.31, 0.50], or absolute supraclavicular skin temperature, Cohen’s d = 0.11, 95% CI [-0.30, 0.52]. Using conditional random forests, we also excluded a variety of alternative explanations for why the method failed to achieve an effect. This protocol of infrared thermography cannot measure BAT thermogenesis and is thus not recommended for future studies to study individual differences in attachment.","PeriodicalId":32922,"journal":{"name":"Social Psychological Bulletin","volume":"171 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-10-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135970101","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Natural hazard versus natural disaster: Does framing the event affect preparedness intentions, attitudes, and behaviour? 自然灾害与自然灾害:事件框架是否会影响准备意图、态度和行为?
Q2 Psychology Pub Date : 2023-10-10 DOI: 10.32872/spb.8357
Lauren J. Vinnell, Taciano L. Milfont, John McClure

Even when perception of risks such as earthquakes is high, preparation is generally low. Previous research shows relatively minor changes in the framing of target issues can impact decisions. In the area of risk, the terms “natural hazards” and “natural disasters” are used inconsistently. Using the Theory of Planned Behaviour as a framework, we conducted an online experiment with a large community sample from Wellington, Aotearoa New Zealand (N = 604) to examine possible framing effects. Half of the participants were asked about their attitudes and intentions regarding preparing for natural hazards and half about natural disasters. We found few significant differences in mean factor scores between the two groups, suggesting that people have attitudes and perceptions of similar strength for both natural hazards and natural disasters. However, half of the factors in structural equation modelling differed in significance or valence between the two conditions and intentions to prepare were positively associated with information-seeking behaviour only for the natural hazards frame, not the natural disaster frame. These findings suggest that the way in which these issues are framed meaningfully impacts how intentions form and translate to actual behaviour. It is possible that participants understood disaster as manifested, devastating impacts of a natural event rather than the potential for impact implied by the term hazard. Such interpretations could influence perceptions of preventability, and therefore preparation. These findings have critical implications for public information campaigns and interventions aimed at increasing preparedness for the risk posed by natural hazards.

即使对地震等风险的认识很高,准备工作也普遍很低。先前的研究表明,目标问题框架的相对微小变化会影响决策。在风险领域,“自然灾害”和“自然灾害”这两个词的使用不一致。以计划行为理论为框架,我们对新西兰惠灵顿的一个大型社区样本(N = 604)进行了一项在线实验,以检验可能的框架效应。一半的参与者被问及他们对准备自然灾害的态度和意图,另一半被问及自然灾害。我们发现两组之间的平均因子得分几乎没有显著差异,这表明人们对自然灾害和自然灾害的态度和看法是相似的。然而,结构方程模型中有一半的因素在两种条件之间的显著性或效价不同,而准备意图仅在自然灾害框架中与信息寻求行为呈正相关,而在自然灾害框架中则没有。这些发现表明,这些问题的框架方式对意图的形成和转化为实际行为的方式产生了有意义的影响。有可能参与者将灾难理解为自然事件所表现出来的破坏性影响,而不是“危险”一词所暗示的潜在影响。这种解释可能影响对可预防性的认识,从而影响准备工作。这些发现对旨在加强对自然灾害所造成的风险的准备工作的宣传运动和干预措施具有重要意义。
{"title":"Natural hazard versus natural disaster: Does framing the event affect preparedness intentions, attitudes, and behaviour?","authors":"Lauren J. Vinnell, Taciano L. Milfont, John McClure","doi":"10.32872/spb.8357","DOIUrl":"https://doi.org/10.32872/spb.8357","url":null,"abstract":"<p xmlns=\"http://www.ncbi.nlm.nih.gov/JATS1\">Even when perception of risks such as earthquakes is high, preparation is generally low. Previous research shows relatively minor changes in the framing of target issues can impact decisions. In the area of risk, the terms “natural hazards” and “natural disasters” are used inconsistently. Using the Theory of Planned Behaviour as a framework, we conducted an online experiment with a large community sample from Wellington, Aotearoa New Zealand (N = 604) to examine possible framing effects. Half of the participants were asked about their attitudes and intentions regarding preparing for natural hazards and half about natural disasters. We found few significant differences in mean factor scores between the two groups, suggesting that people have attitudes and perceptions of similar strength for both natural hazards and natural disasters. However, half of the factors in structural equation modelling differed in significance or valence between the two conditions and intentions to prepare were positively associated with information-seeking behaviour only for the natural hazards frame, not the natural disaster frame. These findings suggest that the way in which these issues are framed meaningfully impacts how intentions form and translate to actual behaviour. It is possible that participants understood disaster as manifested, devastating impacts of a natural event rather than the potential for impact implied by the term hazard. Such interpretations could influence perceptions of preventability, and therefore preparation. These findings have critical implications for public information campaigns and interventions aimed at increasing preparedness for the risk posed by natural hazards.","PeriodicalId":32922,"journal":{"name":"Social Psychological Bulletin","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136357218","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
Social Psychological Bulletin
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1