Advances in Methods and Practices in Psychological Science最新文献_第7页

How Do We Choose Our Giants? Perceptions of Replicability in Psychological Science 我们如何选择我们的巨人?心理科学中的可复制性感知

IF 13.6 1区心理学 Q1 PSYCHOLOGY

Advances in Methods and Practices in Psychological Science

Pub Date : 2021-04-01 DOI: 10.1177/25152459211018199

Manikya Alister, Raine Vickers-Jones, David K. Sewell, T. Ballard

Judgments regarding replicability are vital to scientific progress. The metaphor of “standing on the shoulders of giants” encapsulates the notion that progress is made when new discoveries build on previous findings. Yet attempts to build on findings that are not replicable could mean a great deal of time, effort, and money wasted. In light of the recent “crisis of confidence” in psychological science, the ability to accurately judge the replicability of findings may be more important than ever. In this Registered Report, we examine the factors that influence psychological scientists’ confidence in the replicability of findings. We recruited corresponding authors of articles published in psychology journals between 2014 and 2018 to complete a brief survey in which they were asked to consider 76 specific study attributes that might bear on the replicability of a finding (e.g., preregistration, sample size, statistical methods). Participants were asked to rate the extent to which information regarding each attribute increased or decreased their confidence in the finding being replicated. We examined the extent to which each research attribute influenced average confidence in replicability. We found evidence for six reasonably distinct underlying factors that influenced these judgments and individual differences in the degree to which people’s judgments were influenced by these factors. The conclusions reveal how certain research practices affect other researchers’ perceptions of robustness. We hope our findings will help encourage the use of practices that promote replicability and, by extension, the cumulative progress of psychological science.

关于可复制性的判断对科学进步至关重要。“站在巨人的肩膀上”的比喻概括了这样一个概念，即当新的发现建立在以前的发现之上时，就会取得进步。然而，试图以不可复制的发现为基础，可能意味着浪费大量的时间、精力和金钱。鉴于最近心理科学的“信任危机”，准确判断研究结果的可重复性的能力可能比以往任何时候都更重要。在本注册报告中，我们研究了影响心理科学家对研究结果可复制性的信心的因素。我们招募了2014年至2018年间发表在心理学期刊上的文章的通讯作者来完成一项简短的调查，要求他们考虑76个可能影响研究结果可重复性的特定研究属性(例如，预注册、样本量、统计方法)。参与者被要求对每个属性的信息增加或减少他们对发现被复制的信心的程度进行评级。我们检查了每个研究属性对可复制性平均置信度的影响程度。我们发现了影响这些判断的六个合理而不同的潜在因素的证据，以及人们的判断受这些因素影响程度的个体差异。结论揭示了某些研究实践如何影响其他研究人员对稳健性的看法。我们希望我们的发现将有助于鼓励使用促进可复制性的实践，进而推动心理科学的累积进步。

{"title":"How Do We Choose Our Giants? Perceptions of Replicability in Psychological Science","authors":"Manikya Alister, Raine Vickers-Jones, David K. Sewell, T. Ballard","doi":"10.1177/25152459211018199","DOIUrl":"https://doi.org/10.1177/25152459211018199","url":null,"abstract":"Judgments regarding replicability are vital to scientific progress. The metaphor of “standing on the shoulders of giants” encapsulates the notion that progress is made when new discoveries build on previous findings. Yet attempts to build on findings that are not replicable could mean a great deal of time, effort, and money wasted. In light of the recent “crisis of confidence” in psychological science, the ability to accurately judge the replicability of findings may be more important than ever. In this Registered Report, we examine the factors that influence psychological scientists’ confidence in the replicability of findings. We recruited corresponding authors of articles published in psychology journals between 2014 and 2018 to complete a brief survey in which they were asked to consider 76 specific study attributes that might bear on the replicability of a finding (e.g., preregistration, sample size, statistical methods). Participants were asked to rate the extent to which information regarding each attribute increased or decreased their confidence in the finding being replicated. We examined the extent to which each research attribute influenced average confidence in replicability. We found evidence for six reasonably distinct underlying factors that influenced these judgments and individual differences in the degree to which people’s judgments were influenced by these factors. The conclusions reveal how certain research practices affect other researchers’ perceptions of robustness. We hope our findings will help encourage the use of practices that promote replicability and, by extension, the cumulative progress of psychological science.","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":" ","pages":""},"PeriodicalIF":13.6,"publicationDate":"2021-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/25152459211018199","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44542864","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 6

Multilab Direct Replication of Flavell, Beach, and Chinsky (1966): Spontaneous Verbal Rehearsal in a Memory Task as a Function of Age. Flavell, Beach和Chinsky(1966)的多实验室直接复制:记忆任务中的自发言语排练作为年龄的函数

IF 13.4 1区心理学 Q1 PSYCHOLOGY

Advances in Methods and Practices in Psychological Science

Pub Date : 2021-04-01 Epub Date: 2021-06-24 DOI: 10.1177/25152459211018187

Emily M Elliott, Candice C Morey, Angela M AuBuchon, Nelson Cowan, Chris Jarrold, Eryn Adams, Meg Attwood, Büşra Bayram, Stefen Beeler-Duden, Taran Y Blakstvedt, Gerhard Büttner, Thomas Castelain, Shari Cave, Davide Crepaldi, Eivor Fredriksen, Bret Glass, Andrew Graves, Dominic Guitard, Stefanie Hoehl, Alexis Hosch, Stéphanie Jeanneret, Tanya N Joseph, Chris Koch, Jaroslaw R Lelonkiewicz, Gary Lupyan, Amalia McDonald, Grace Meissner, Whitney Mendenhall, David Moreau, Thomas Ostermann, Asil Ali Özdoğru, Francesca Padovani, Sebastian Poloczek, Jan Phillip Röer, Christina Schonberg, Christian K Tamnes, Martin J Tomasik, Beatrice Valentini, Evie Vergauwe, Haley Vlach, Martin Voracek

Work by Flavell, Beach, and Chinsky (1966) indicated a change in the spontaneous production of overt verbalization behaviors when comparing young children (age 5) to older children (age 10). Despite the critical role that this evidence of a change in verbalization behaviors plays in modern theories of cognitive development and working memory, there has only been one other published near-replication of this work (Keeney, Cannizzo, & Flavell, 1967). This Registered Replication Report relied upon 17 labs who contributed their results to a larger and more comprehensive sample of children. We assessed memory performance and the presence or absence of verbalization behaviors of young children at different ages, and determined that the original pattern of findings was largely upheld: older children were more likely to verbalize, and their memory spans improved. We confirmed that 5- and 6-year-old children who verbalized recalled more than children who did not verbalize. However, unlike Flavell et al., substantial proportions of our 5- and 6-year-old samples overtly verbalized at least sometimes during the picture memory task. Also, continuous increase in overt verbalization from 7- to 10-years-old was not consistently evident in our samples. These robust findings should be weighed when considering theories of cognitive development, particularly theories concerning when verbal rehearsal emerges and relations between speech and memory.

Flavell、Beach和Chinsky的研究表明，在比较幼儿（5岁）和年长儿童（10岁）时，明显言语行为的自发产生发生了变化。尽管这一言语行为变化的证据在现代认知发展和工作记忆理论中发挥着关键作用，但只有一篇发表的论文接近于复制这项工作。在这份注册复制报告中，我们依赖于来自17个实验室的研究人员，他们将自己的结果贡献给了更大、更全面的儿童样本。我们评估了不同年龄幼儿的记忆表现和言语行为的存在与否，并确定最初的发现模式在很大程度上得到了支持：年龄较大的儿童更有可能言语，他们的记忆广度也有所提高。我们证实，5岁和6岁的用言语表达的儿童比不用言语的儿童回忆得更多。然而，与Flavell等人不同的是，我们5岁和6岁的样本中有相当一部分在图片记忆任务中公开表达，至少有时是这样。此外，在我们的样本中，从7岁到10岁，显性言语的持续增加并不总是明显的。在考虑认知发展理论时，应该权衡这些有力的发现，特别是关于言语排练何时出现以及言语和记忆之间关系的理论。

{"title":"Multilab Direct Replication of Flavell, Beach, and Chinsky (1966): Spontaneous Verbal Rehearsal in a Memory Task as a Function of Age.","authors":"Emily M Elliott, Candice C Morey, Angela M AuBuchon, Nelson Cowan, Chris Jarrold, Eryn Adams, Meg Attwood, Büşra Bayram, Stefen Beeler-Duden, Taran Y Blakstvedt, Gerhard Büttner, Thomas Castelain, Shari Cave, Davide Crepaldi, Eivor Fredriksen, Bret Glass, Andrew Graves, Dominic Guitard, Stefanie Hoehl, Alexis Hosch, Stéphanie Jeanneret, Tanya N Joseph, Chris Koch, Jaroslaw R Lelonkiewicz, Gary Lupyan, Amalia McDonald, Grace Meissner, Whitney Mendenhall, David Moreau, Thomas Ostermann, Asil Ali Özdoğru, Francesca Padovani, Sebastian Poloczek, Jan Phillip Röer, Christina Schonberg, Christian K Tamnes, Martin J Tomasik, Beatrice Valentini, Evie Vergauwe, Haley Vlach, Martin Voracek","doi":"10.1177/25152459211018187","DOIUrl":"10.1177/25152459211018187","url":null,"abstract":"Work by Flavell, Beach, and Chinsky (1966) indicated a change in the spontaneous production of overt verbalization behaviors when comparing young children (age 5) to older children (age 10). Despite the critical role that this evidence of a change in verbalization behaviors plays in modern theories of cognitive development and working memory, there has only been one other published near-replication of this work (Keeney, Cannizzo, & Flavell, 1967). This Registered Replication Report relied upon 17 labs who contributed their results to a larger and more comprehensive sample of children. We assessed memory performance and the presence or absence of verbalization behaviors of young children at different ages, and determined that the original pattern of findings was largely upheld: older children were more likely to verbalize, and their memory spans improved. We confirmed that 5- and 6-year-old children who verbalized recalled more than children who did not verbalize. However, unlike Flavell et al., substantial proportions of our 5- and 6-year-old samples overtly verbalized at least sometimes during the picture memory task. Also, continuous increase in overt verbalization from 7- to 10-years-old was not consistently evident in our samples. These robust findings should be weighed when considering theories of cognitive development, particularly theories concerning when verbal rehearsal emerges and relations between speech and memory.","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":" ","pages":""},"PeriodicalIF":13.4,"publicationDate":"2021-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12349909/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42721355","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Leveraging Containers for Reproducible Psychological Research 利用容器进行可重复的心理学研究

IF 13.6 1区心理学 Q1 PSYCHOLOGY

Advances in Methods and Practices in Psychological Science

Pub Date : 2021-04-01 DOI: 10.1177/25152459211017853

K. Wiebels, David Moreau

Containers have become increasingly popular in computing and software engineering and are gaining traction in scientific research. They allow packaging up all code and dependencies to ensure that analyses run reliably across a range of operating systems and software versions. Despite being a crucial component for reproducible science, containerization has yet to become mainstream in psychology. In this tutorial, we describe the logic behind containers, what they are, and the practical problems they can solve. We walk the reader through the implementation of containerization within a research workflow with examples using Docker and R. Specifically, we describe how to use existing containers, build personalized containers, and share containers alongside publications. We provide a worked example that includes all steps required to set up a container for a research project and can easily be adapted and extended. We conclude with a discussion of the possibilities afforded by the large-scale adoption of containerization, especially in the context of cumulative, open science, toward a more efficient and inclusive research ecosystem.

容器在计算和软件工程中越来越受欢迎，在科学研究中也越来越受关注。它们允许打包所有代码和依赖项，以确保分析在一系列操作系统和软件版本中可靠运行。尽管容器化是可复制科学的重要组成部分，但它尚未成为心理学的主流。在本教程中，我们将描述容器背后的逻辑、它们是什么以及它们可以解决的实际问题。我们以Docker和R为例，带领读者了解研究工作流程中容器化的实现。具体而言，我们描述了如何使用现有容器、构建个性化容器以及与出版物一起共享容器。我们提供了一个工作示例，其中包括为研究项目设置容器所需的所有步骤，并且可以很容易地进行调整和扩展。最后，我们讨论了大规模采用集装箱化所带来的可能性，特别是在累积、开放科学的背景下，建立一个更高效、更具包容性的研究生态系统。

{"title":"Leveraging Containers for Reproducible Psychological Research","authors":"K. Wiebels, David Moreau","doi":"10.1177/25152459211017853","DOIUrl":"https://doi.org/10.1177/25152459211017853","url":null,"abstract":"Containers have become increasingly popular in computing and software engineering and are gaining traction in scientific research. They allow packaging up all code and dependencies to ensure that analyses run reliably across a range of operating systems and software versions. Despite being a crucial component for reproducible science, containerization has yet to become mainstream in psychology. In this tutorial, we describe the logic behind containers, what they are, and the practical problems they can solve. We walk the reader through the implementation of containerization within a research workflow with examples using Docker and R. Specifically, we describe how to use existing containers, build personalized containers, and share containers alongside publications. We provide a worked example that includes all steps required to set up a container for a research project and can easily be adapted and extended. We conclude with a discussion of the possibilities afforded by the large-scale adoption of containerization, especially in the context of cumulative, open science, toward a more efficient and inclusive research ecosystem.","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":" ","pages":""},"PeriodicalIF":13.6,"publicationDate":"2021-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/25152459211017853","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47832005","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 10

Citation Patterns Following a Strongly Contradictory Replication Result: Four Case Studies From Psychology 引用模式遵循强烈矛盾的复制结果:来自心理学的四个案例研究

IF 13.6 1区心理学 Q1 PSYCHOLOGY

Advances in Methods and Practices in Psychological Science

Pub Date : 2021-02-09 DOI: 10.1177/25152459211040837

T. Hardwicke, Dénes Szűcs, Robert T. Thibault, S. Crüwell, O. R. van den Akker, Michèle B. Nuijten, J. Ioannidis

Replication studies that contradict prior findings may facilitate scientific self-correction by triggering a reappraisal of the original studies; however, the research community’s response to replication results has not been studied systematically. One approach for gauging responses to replication results is to examine how they affect citations to original studies. In this study, we explored postreplication citation patterns in the context of four prominent multilaboratory replication attempts published in the field of psychology that strongly contradicted and outweighed prior findings. Generally, we observed a small postreplication decline in the number of favorable citations and a small increase in unfavorable citations. This indicates only modest corrective effects and implies considerable perpetuation of belief in the original findings. Replication results that strongly contradict an original finding do not necessarily nullify its credibility; however, one might at least expect the replication results to be acknowledged and explicitly debated in subsequent literature. By contrast, we found substantial citation bias: The majority of articles citing the original studies neglected to cite relevant replication results. Of those articles that did cite the replication but continued to cite the original study favorably, approximately half offered an explicit defense of the original study. Our findings suggest that even replication results that strongly contradict original findings do not necessarily prompt a corrective response from the research community.

与先前研究结果相矛盾的重复性研究可能会引发对原始研究的重新评估，从而促进科学自我纠正;然而，研究界对重复性结果的反应尚未得到系统的研究。衡量对复制结果的反应的一种方法是检查它们如何影响对原始研究的引用。在这项研究中，我们在心理学领域发表的四篇著名的多实验室复制论文中探索了复制后的引用模式，这些论文与先前的研究结果强烈矛盾。一般来说，我们观察到有利引用的数量在复制后略有下降，而不利引用的数量略有增加。这表明只有适度的纠正效果，并意味着对原始发现的相当持久的信念。与原始发现强烈矛盾的重复结果不一定会使其可信度无效;然而，人们至少可以期望复制结果在随后的文献中得到承认和明确的辩论。相比之下，我们发现了大量的引用偏倚:大多数引用原始研究的文章忽略了引用相关的复制结果。在那些引用了重复实验但继续引用原始研究的文章中，大约一半的文章为原始研究提供了明确的辩护。我们的研究结果表明，即使是与原始研究结果强烈矛盾的复制结果，也不一定会引起研究界的纠正反应。

{"title":"Citation Patterns Following a Strongly Contradictory Replication Result: Four Case Studies From Psychology","authors":"T. Hardwicke, Dénes Szűcs, Robert T. Thibault, S. Crüwell, O. R. van den Akker, Michèle B. Nuijten, J. Ioannidis","doi":"10.1177/25152459211040837","DOIUrl":"https://doi.org/10.1177/25152459211040837","url":null,"abstract":"Replication studies that contradict prior findings may facilitate scientific self-correction by triggering a reappraisal of the original studies; however, the research community’s response to replication results has not been studied systematically. One approach for gauging responses to replication results is to examine how they affect citations to original studies. In this study, we explored postreplication citation patterns in the context of four prominent multilaboratory replication attempts published in the field of psychology that strongly contradicted and outweighed prior findings. Generally, we observed a small postreplication decline in the number of favorable citations and a small increase in unfavorable citations. This indicates only modest corrective effects and implies considerable perpetuation of belief in the original findings. Replication results that strongly contradict an original finding do not necessarily nullify its credibility; however, one might at least expect the replication results to be acknowledged and explicitly debated in subsequent literature. By contrast, we found substantial citation bias: The majority of articles citing the original studies neglected to cite relevant replication results. Of those articles that did cite the replication but continued to cite the original study favorably, approximately half offered an explicit defense of the original study. Our findings suggest that even replication results that strongly contradict original findings do not necessarily prompt a corrective response from the research community.","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":" ","pages":""},"PeriodicalIF":13.6,"publicationDate":"2021-02-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48150510","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 15

That’s a Lot to Process! Pitfalls of Popular Path Models 这是一个很大的过程!流行路径模型的缺陷

IF 13.6 1区心理学 Q1 PSYCHOLOGY

Advances in Methods and Practices in Psychological Science

Pub Date : 2021-02-01 DOI: 10.1177/25152459221095827

J. Rohrer, Paul Hünermund, Ruben C. Arslan, M. Elson

Path models to test claims about mediation and moderation are a staple of psychology. But applied researchers may sometimes not understand the underlying causal inference problems and thus endorse conclusions that rest on unrealistic assumptions. In this article, we aim to provide a clear explanation for the limited conditions under which standard procedures for mediation and moderation analysis can succeed. We discuss why reversing arrows or comparing model fit indices cannot tell us which model is the right one and how tests of conditional independence can at least tell us where our model goes wrong. Causal modeling practices in psychology are far from optimal but may be kept alive by domain norms that demand every article makes some novel claim about processes and boundary conditions. We end with a vision for a different research culture in which causal inference is pursued in a much slower, more deliberate, and collaborative manner.

测试调解和适度主张的路径模型是心理学的主要内容。但应用研究人员有时可能不理解潜在的因果推断问题，从而认可基于不切实际假设的结论。在这篇文章中，我们的目的是为调解和适度分析的标准程序能够成功的有限条件提供一个明确的解释。我们讨论了为什么反向箭头或比较模型拟合指数不能告诉我们哪个模型是正确的，以及条件独立性测试如何至少告诉我们我们的模型哪里出了问题。心理学中的因果建模实践远不是最优的，但可以通过领域规范来保持活力，这些规范要求每一篇文章都对过程和边界条件提出一些新颖的主张。最后，我们展望了一种不同的研究文化，在这种文化中，因果推理以一种更慢、更深思熟虑、更协作的方式进行。

引用次数: 63

A Traveler’s Guide to the Multiverse: Promises, Pitfalls, and a Framework for the Evaluation of Analytic Decisions 多元宇宙旅行者指南：承诺、陷阱和分析决策评估框架

IF 13.6 1区心理学 Q1 PSYCHOLOGY

Advances in Methods and Practices in Psychological Science

Pub Date : 2021-01-01 DOI: 10.1177/2515245920954925

M. Del Giudice, S. Gangestad

Decisions made by researchers while analyzing data (e.g., how to measure variables, how to handle outliers) are sometimes arbitrary, without an objective justification for choosing one alternative over another. Multiverse-style methods (e.g., specification curve, vibration of effects) estimate an effect across an entire set of possible specifications to expose the impact of hidden degrees of freedom and/or obtain robust, less biased estimates of the effect of interest. However, if specifications are not truly arbitrary, multiverse-style analyses can produce misleading results, potentially hiding meaningful effects within a mass of poorly justified alternatives. So far, a key question has received scant attention: How does one decide whether alternatives are arbitrary? We offer a framework and conceptual tools for doing so. We discuss three kinds of a priori nonequivalence among alternatives—measurement nonequivalence, effect nonequivalence, and power/precision nonequivalence. The criteria we review lead to three decision scenarios: Type E decisions (principled equivalence), Type N decisions (principled nonequivalence), and Type U decisions (uncertainty). In uncertain scenarios, multiverse-style analysis should be conducted in a deliberately exploratory fashion. The framework is discussed with reference to published examples and illustrated with the help of a simulated data set. Our framework will help researchers reap the benefits of multiverse-style methods while avoiding their pitfalls.

研究人员在分析数据时做出的决定（例如，如何测量变量，如何处理异常值）有时是武断的，没有客观的理由选择一种替代方案。多宇宙风格的方法（例如，规格曲线、效果振动）估计整个可能规格集的效果，以暴露隐藏自由度的影响和/或获得对感兴趣效果的稳健、偏差较小的估计。然而，如果规范不是真正武断的，多元宇宙风格的分析可能会产生误导性的结果，可能会在大量理由不足的替代方案中隐藏有意义的效果。到目前为止，一个关键问题很少受到关注：如何决定替代方案是否武断？我们提供了这样做的框架和概念工具。我们讨论了替代方案中的三种先验非等价性——测量非等价性、效应非等价性和功率/精度非等价性。我们审查的标准导致了三种决策场景：E型决策（原则等价）、N型决策（原理非等价）和U型决策（不确定性）。在不确定的情况下，多元宇宙风格的分析应该以一种有意探索的方式进行。该框架参考已发表的实例进行了讨论，并借助模拟数据集进行了说明。我们的框架将帮助研究人员获得多元宇宙风格方法的好处，同时避免它们的陷阱。

{"title":"A Traveler’s Guide to the Multiverse: Promises, Pitfalls, and a Framework for the Evaluation of Analytic Decisions","authors":"M. Del Giudice, S. Gangestad","doi":"10.1177/2515245920954925","DOIUrl":"https://doi.org/10.1177/2515245920954925","url":null,"abstract":"Decisions made by researchers while analyzing data (e.g., how to measure variables, how to handle outliers) are sometimes arbitrary, without an objective justification for choosing one alternative over another. Multiverse-style methods (e.g., specification curve, vibration of effects) estimate an effect across an entire set of possible specifications to expose the impact of hidden degrees of freedom and/or obtain robust, less biased estimates of the effect of interest. However, if specifications are not truly arbitrary, multiverse-style analyses can produce misleading results, potentially hiding meaningful effects within a mass of poorly justified alternatives. So far, a key question has received scant attention: How does one decide whether alternatives are arbitrary? We offer a framework and conceptual tools for doing so. We discuss three kinds of a priori nonequivalence among alternatives—measurement nonequivalence, effect nonequivalence, and power/precision nonequivalence. The criteria we review lead to three decision scenarios: Type E decisions (principled equivalence), Type N decisions (principled nonequivalence), and Type U decisions (uncertainty). In uncertain scenarios, multiverse-style analysis should be conducted in a deliberately exploratory fashion. The framework is discussed with reference to published examples and illustrated with the help of a simulated data set. Our framework will help researchers reap the benefits of multiverse-style methods while avoiding their pitfalls.","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":" ","pages":""},"PeriodicalIF":13.6,"publicationDate":"2021-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/2515245920954925","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46424382","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 55

A multi-lab study of bilingual infants: Exploring the preference for infant-directed speech. 双语婴儿的多实验室研究：探索婴儿引导言语的偏好。

IF 15.6 1区心理学 Q1 PSYCHOLOGY

Advances in Methods and Practices in Psychological Science

Pub Date : 2021-01-01 Epub Date: 2021-03-12 DOI: 10.1177/2515245920974622

Krista Byers-Heinlein, Angeline Sin Mei Tsui, Christina Bergmann, Alexis K Black, Anna Brown, Maria Julia Carbajal, Samantha Durrant, Christopher T Fennell, Anne-Caroline Fiévet, Michael C Frank, Anja Gampe, Judit Gervain, Nayeli Gonzalez-Gomez, J Kiley Hamlin, Naomi Havron, Mikołaj Hernik, Shila Kerr, Hilary Killam, Kelsey Klassen, Jessica E Kosie, Ágnes Melinda Kovács, Casey Lew-Williams, Liquan Liu, Nivedita Mani, Caterina Marino, Meghan Mastroberardino, Victoria Mateu, Claire Noble, Adriel John Orena, Linda Polka, Christine E Potter, Melanie Schreiner, Leher Singh, Melanie Soderstrom, Megha Sundara, Connor Waddell, Janet F Werker, Stephanie Wermelinger

From the earliest months of life, infants prefer listening to and learn better from infant-directed speech (IDS) than adult-directed speech (ADS). Yet, IDS differs within communities, across languages, and across cultures, both in form and in prevalence. This large-scale, multi-site study used the diversity of bilingual infant experiences to explore the impact of different types of linguistic experience on infants' IDS preference. As part of the multi-lab ManyBabies 1 project, we compared lab-matched samples of 333 bilingual and 385 monolingual infants' preference for North-American English IDS (cf. ManyBabies Consortium, 2020: ManyBabies 1), tested in 17 labs in 7 countries. Those infants were tested in two age groups: 6-9 months (the younger sample) and 12-15 months (the older sample). We found that bilingual and monolingual infants both preferred IDS to ADS, and did not differ in terms of the overall magnitude of this preference. However, amongst bilingual infants who were acquiring North-American English (NAE) as a native language, greater exposure to NAE was associated with a stronger IDS preference, extending the previous finding from ManyBabies 1 that monolinguals learning NAE as a native language showed a stronger preference than infants unexposed to NAE. Together, our findings indicate that IDS preference likely makes a similar contribution to monolingual and bilingual development, and that infants are exquisitely sensitive to the nature and frequency of different types of language input in their early environments.

从婴儿出生后的最初几个月开始，他们就更喜欢听婴儿引导式言语（IDS），而且从婴儿引导式言语（IDS）中学习的效果比成人引导式言语（ADS）更好。然而，在不同社区、不同语言和不同文化背景下，婴儿引导式言语在形式和流行程度上都存在差异。这项大规模、多地点的研究利用婴儿双语体验的多样性来探讨不同类型的语言体验对婴儿IDS偏好的影响。作为多实验室 "ManyBabies 1 "项目的一部分，我们比较了 333 名双语婴儿和 385 名单语婴儿的实验室匹配样本对北美英语 IDS 的偏好（参见 ManyBabies Consortium, 2020: ManyBabies 1），这些样本在 7 个国家的 17 个实验室进行了测试。这些婴儿分两个年龄组进行测试：6-9 个月（较小的样本）和 12-15 个月（较大的样本）。我们发现，与 ADS 相比，双语婴儿和单语婴儿都更喜欢 IDS，而且在总体偏好程度上没有差异。然而，在将北美英语（NAE）作为母语学习的双语婴儿中，接触北美英语越多的婴儿越偏好IDS，这延续了ManyBabies 1之前的发现，即将北美英语作为母语学习的单语婴儿比未接触北美英语的婴儿表现出更强的偏好。总之，我们的研究结果表明，IDS偏好可能对单语和双语的发展具有类似的作用，而且婴儿对早期环境中不同类型语言输入的性质和频率非常敏感。

{"title":"A multi-lab study of bilingual infants: Exploring the preference for infant-directed speech.","authors":"Krista Byers-Heinlein, Angeline Sin Mei Tsui, Christina Bergmann, Alexis K Black, Anna Brown, Maria Julia Carbajal, Samantha Durrant, Christopher T Fennell, Anne-Caroline Fiévet, Michael C Frank, Anja Gampe, Judit Gervain, Nayeli Gonzalez-Gomez, J Kiley Hamlin, Naomi Havron, Mikołaj Hernik, Shila Kerr, Hilary Killam, Kelsey Klassen, Jessica E Kosie, Ágnes Melinda Kovács, Casey Lew-Williams, Liquan Liu, Nivedita Mani, Caterina Marino, Meghan Mastroberardino, Victoria Mateu, Claire Noble, Adriel John Orena, Linda Polka, Christine E Potter, Melanie Schreiner, Leher Singh, Melanie Soderstrom, Megha Sundara, Connor Waddell, Janet F Werker, Stephanie Wermelinger","doi":"10.1177/2515245920974622","DOIUrl":"10.1177/2515245920974622","url":null,"abstract":"From the earliest months of life, infants prefer listening to and learn better from infant-directed speech (IDS) than adult-directed speech (ADS). Yet, IDS differs within communities, across languages, and across cultures, both in form and in prevalence. This large-scale, multi-site study used the diversity of bilingual infant experiences to explore the impact of different types of linguistic experience on infants' IDS preference. As part of the multi-lab ManyBabies 1 project, we compared lab-matched samples of 333 bilingual and 385 monolingual infants' preference for North-American English IDS (cf. ManyBabies Consortium, 2020: ManyBabies 1), tested in 17 labs in 7 countries. Those infants were tested in two age groups: 6-9 months (the younger sample) and 12-15 months (the older sample). We found that bilingual and monolingual infants both preferred IDS to ADS, and did not differ in terms of the overall magnitude of this preference. However, amongst bilingual infants who were acquiring North-American English (NAE) as a native language, greater exposure to NAE was associated with a stronger IDS preference, extending the previous finding from ManyBabies 1 that monolinguals learning NAE as a native language showed a stronger preference than infants unexposed to NAE. Together, our findings indicate that IDS preference likely makes a similar contribution to monolingual and bilingual development, and that infants are exquisitely sensitive to the nature and frequency of different types of language input in their early environments.","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":"4 1","pages":""},"PeriodicalIF":15.6,"publicationDate":"2021-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9273003/pdf/nihms-1769134.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"40497206","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Acknowledgment 鸣谢

IF 13.6 1区心理学 Q1 PSYCHOLOGY

Advances in Methods and Practices in Psychological Science

Pub Date : 2021-01-01 DOI: 10.1177/2515245921993161

引用次数: 0

Analyzing Individual Differences in Intervention-Related Changes 干预相关改变的个体差异分析

IF 13.6 1区心理学 Q1 PSYCHOLOGY

Advances in Methods and Practices in Psychological Science

Pub Date : 2021-01-01 DOI: 10.1177/2515245920979172

Tanja Könen, J. Karbach

Intervention studies can be expensive and time-consuming, which is why it is important to extract as much knowledge as possible. We discuss benefits and limitations of analyzing individual differences in intervention studies in addition to traditional analyses of average group effects. First, we present a short introduction to latent change modeling and measurement invariance in the context of intervention studies. Then, we give an overview on options for analyzing individual differences in intervention-related changes with a focus on how substantive information can be distinguished from methodological artifacts (e.g., regression to the mean). The main topics are benefits and limitations of predicting changes with baseline data and of analyzing correlated change. Both approaches can offer descriptive correlational information about individuals in interventions, which can inform future variations of experimental conditions. Applications increasingly emerge in the literature—from clinical, developmental, and educational psychology to occupational psychology—and demonstrate their potential across all of psychology.

干预研究可能既昂贵又耗时，这就是为什么提取尽可能多的知识很重要。除了传统的平均组效应分析外，我们还讨论了在干预研究中分析个体差异的益处和局限性。首先，我们简要介绍了干预研究背景下的潜在变化建模和测量不变性。然后，我们概述了在干预相关变化中分析个体差异的选择，重点是如何将实质性信息与方法学上的人为因素(例如，回归均值)区分开来。主要的主题是利用基线数据预测变化和分析相关变化的好处和局限性。这两种方法都可以提供干预中个体的描述性相关信息，这可以为实验条件的未来变化提供信息。越来越多的应用出现在文献中——从临床心理学、发展心理学、教育心理学到职业心理学——并展示了它们在所有心理学领域的潜力。

{"title":"Analyzing Individual Differences in Intervention-Related Changes","authors":"Tanja Könen, J. Karbach","doi":"10.1177/2515245920979172","DOIUrl":"https://doi.org/10.1177/2515245920979172","url":null,"abstract":"Intervention studies can be expensive and time-consuming, which is why it is important to extract as much knowledge as possible. We discuss benefits and limitations of analyzing individual differences in intervention studies in addition to traditional analyses of average group effects. First, we present a short introduction to latent change modeling and measurement invariance in the context of intervention studies. Then, we give an overview on options for analyzing individual differences in intervention-related changes with a focus on how substantive information can be distinguished from methodological artifacts (e.g., regression to the mean). The main topics are benefits and limitations of predicting changes with baseline data and of analyzing correlated change. Both approaches can offer descriptive correlational information about individuals in interventions, which can inform future variations of experimental conditions. Applications increasingly emerge in the literature—from clinical, developmental, and educational psychology to occupational psychology—and demonstrate their potential across all of psychology.","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":" ","pages":""},"PeriodicalIF":13.6,"publicationDate":"2021-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/2515245920979172","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46600686","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 5

An Introduction to Linear Mixed-Effects Modeling in R R中线性混合效果建模简介

IF 13.6 1区心理学 Q1 PSYCHOLOGY

Advances in Methods and Practices in Psychological Science

Pub Date : 2021-01-01 DOI: 10.1177/2515245920960351

V. Brown

This Tutorial serves as both an approachable theoretical introduction to mixed-effects modeling and a practical introduction to how to implement mixed-effects models in R. The intended audience is researchers who have some basic statistical knowledge, but little or no experience implementing mixed-effects models in R using their own data. In an attempt to increase the accessibility of this Tutorial, I deliberately avoid using mathematical terminology beyond what a student would learn in a standard graduate-level statistics course, but I reference articles and textbooks that provide more detail for interested readers. This Tutorial includes snippets of R code throughout; the data and R script used to build the models described in the text are available via OSF at https://osf.io/v6qag/, so readers can follow along if they wish. The goal of this practical introduction is to provide researchers with the tools they need to begin implementing mixed-effects models in their own research.

本教程既是混合效果建模的理论介绍，也是如何在R中实现混合效果模型的实用介绍。目标受众是具有一些基本统计知识，但很少或没有使用自己的数据在R中实现混合效果模型的经验的研究人员。为了增加本教程的可访问性，我故意避免使用学生在标准研究生水平的统计学课程中学习到的数学术语，但我参考了为感兴趣的读者提供更多细节的文章和教科书。本教程包括整个R代码片段;用于构建文本中描述的模型的数据和R脚本可通过OSF在https://osf.io/v6qag/上获得，因此读者可以按照他们的意愿进行操作。本实用介绍的目的是为研究人员提供他们在自己的研究中开始实施混合效应模型所需的工具。

引用次数: 138