Pub Date : 2023-04-01Epub Date: 2023-04-18DOI: 10.1177/25152459221147250
Yotam Erel, Katherine Adams Shannon, Junyi Chu, Kim Scott, Melissa Kline Struhl, Peng Cao, Xincheng Tan, Peter Hart, Gal Raz, Sabrina Piccolo, Catherine Mei, Christine Potter, Sagi Jaffe-Dax, Casey Lew-Williams, Joshua Tenenbaum, Katherine Fairchild, Amit Bermano, Shari Liu
Technological advances in psychological research have enabled large-scale studies of human behavior and streamlined pipelines for automatic processing of data. However, studies of infants and children have not fully reaped these benefits because the behaviors of interest, such as gaze duration and direction, still have to be extracted from video through a laborious process of manual annotation, even when these data are collected online. Recent advances in computer vision raise the possibility of automated annotation of these video data. In this article, we built on a system for automatic gaze annotation in young children, iCatcher, by engineering improvements and then training and testing the system (referred to hereafter as iCatcher+) on three data sets with substantial video and participant variability (214 videos collected in U.S. lab and field sites, 143 videos collected in Senegal field sites, and 265 videos collected via webcams in homes; participant age range = 4 months-3.5 years). When trained on each of these data sets, iCatcher+ performed with near human-level accuracy on held-out videos on distinguishing "LEFT" versus "RIGHT" and "ON" versus "OFF" looking behavior across all data sets. This high performance was achieved at the level of individual frames, experimental trials, and study videos; held across participant demographics (e.g., age, race/ethnicity), participant behavior (e.g., movement, head position), and video characteristics (e.g., luminance); and generalized to a fourth, entirely held-out online data set. We close by discussing next steps required to fully automate the life cycle of online infant and child behavioral studies, representing a key step toward enabling robust and high-throughput developmental research.
{"title":"iCatcher+: Robust and Automated Annotation of Infants' and Young Children's Gaze Behavior From Videos Collected in Laboratory, Field, and Online Studies.","authors":"Yotam Erel, Katherine Adams Shannon, Junyi Chu, Kim Scott, Melissa Kline Struhl, Peng Cao, Xincheng Tan, Peter Hart, Gal Raz, Sabrina Piccolo, Catherine Mei, Christine Potter, Sagi Jaffe-Dax, Casey Lew-Williams, Joshua Tenenbaum, Katherine Fairchild, Amit Bermano, Shari Liu","doi":"10.1177/25152459221147250","DOIUrl":"10.1177/25152459221147250","url":null,"abstract":"<p><p>Technological advances in psychological research have enabled large-scale studies of human behavior and streamlined pipelines for automatic processing of data. However, studies of infants and children have not fully reaped these benefits because the behaviors of interest, such as gaze duration and direction, still have to be extracted from video through a laborious process of manual annotation, even when these data are collected online. Recent advances in computer vision raise the possibility of automated annotation of these video data. In this article, we built on a system for automatic gaze annotation in young children, iCatcher, by engineering improvements and then training and testing the system (referred to hereafter as iCatcher+) on three data sets with substantial video and participant variability (214 videos collected in U.S. lab and field sites, 143 videos collected in Senegal field sites, and 265 videos collected via webcams in homes; participant age range = 4 months-3.5 years). When trained on each of these data sets, iCatcher+ performed with near human-level accuracy on held-out videos on distinguishing \"LEFT\" versus \"RIGHT\" and \"ON\" versus \"OFF\" looking behavior across all data sets. This high performance was achieved at the level of individual frames, experimental trials, and study videos; held across participant demographics (e.g., age, race/ethnicity), participant behavior (e.g., movement, head position), and video characteristics (e.g., luminance); and generalized to a fourth, entirely held-out online data set. We close by discussing next steps required to fully automate the life cycle of online infant and child behavioral studies, representing a key step toward enabling robust and high-throughput developmental research.</p>","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":"6 2","pages":""},"PeriodicalIF":13.6,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/55/6e/nihms-1916587.PMC10471135.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10152159","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-04-01DOI: 10.1177/25152459231177713
Yngwie Asbjørn Nielsen, Isabel Thielmann, Stefan Pfattheicher
Two participants completing a psychometric scale may leave wildly different responses yet attain the same mean score. Moreover, the mean score often does not represent the bulk of participants’ responses, which may be skewed, kurtotic, or bimodal. Even so, researchers in psychological science often aggregate item scores using an unweighted mean or a sum score, thereby neglecting a substantial amount of information. In the present contribution, we explore whether other summary statistics of a scale (e.g., the standard deviation, the median, or the kurtosis) can capture and leverage some of this neglected information to improve prediction of a broad range of outcome measures: life satisfaction, mental health, self-esteem, counterproductive work behavior, and social value orientation. Overall, across 32 psychometric scales and three data sets (total N = 8,376), we show that the mean is the strongest predictor of all five outcomes considered, with little to no additional variance explained by other summary statistics. These results provide justification for the current practice of relying on the mean score but hopefully inspire future research to explore the predictive power of other summary statistics for relevant outcomes. For this purpose, we provide a tutorial and example code for R.
{"title":"Beyond the Mean: Can We Improve the Predictive Power of Psychometric Scales?","authors":"Yngwie Asbjørn Nielsen, Isabel Thielmann, Stefan Pfattheicher","doi":"10.1177/25152459231177713","DOIUrl":"https://doi.org/10.1177/25152459231177713","url":null,"abstract":"Two participants completing a psychometric scale may leave wildly different responses yet attain the same mean score. Moreover, the mean score often does not represent the bulk of participants’ responses, which may be skewed, kurtotic, or bimodal. Even so, researchers in psychological science often aggregate item scores using an unweighted mean or a sum score, thereby neglecting a substantial amount of information. In the present contribution, we explore whether other summary statistics of a scale (e.g., the standard deviation, the median, or the kurtosis) can capture and leverage some of this neglected information to improve prediction of a broad range of outcome measures: life satisfaction, mental health, self-esteem, counterproductive work behavior, and social value orientation. Overall, across 32 psychometric scales and three data sets (total N = 8,376), we show that the mean is the strongest predictor of all five outcomes considered, with little to no additional variance explained by other summary statistics. These results provide justification for the current practice of relying on the mean score but hopefully inspire future research to explore the predictive power of other summary statistics for relevant outcomes. For this purpose, we provide a tutorial and example code for R.","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":" ","pages":""},"PeriodicalIF":13.6,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46022858","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-04-01DOI: 10.1177/25152459231156085
Zachary J. Kunicki, Meghan L. Smith, E. Murray
Many psychological researchers use some form of a visual diagram in their research processes. Model diagrams used with structural equation models (SEMs) and causal directed acyclic graphs (DAGs) can guide causal-inference research. SEM diagrams and DAGs share visual similarities, often leading researchers familiar with one to wonder how the other differs. This article is intended to serve as a guide for researchers in the psychological sciences and psychiatric epidemiology on the distinctions between these methods. We offer high-level overviews of SEMs and causal DAGs using a guiding example. We then compare and contrast the two methodologies and describe when each would be used. In brief, SEM diagrams are both a conceptual and statistical tool in which a model is drawn and then tested, whereas causal DAGs are exclusively conceptual tools used to help guide researchers in developing an analytic strategy and interpreting results. Causal DAGs are explicitly tools for causal inference, whereas the results of a SEM are only sometimes interpreted causally. A DAG may be thought of as a “qualitative schematic” for some SEMs, whereas SEMs may be thought of as an “algebraic system” for a causal DAG. As psychology begins to adopt more causal-modeling concepts and psychiatric epidemiology begins to adopt more latent-variable concepts, the ability of researchers to understand and possibly combine both of these tools is valuable. Using an applied example, we provide sample analyses, code, and write-ups for both SEM and causal DAG approaches.
{"title":"A Primer on Structural Equation Model Diagrams and Directed Acyclic Graphs: When and How to Use Each in Psychological and Epidemiological Research","authors":"Zachary J. Kunicki, Meghan L. Smith, E. Murray","doi":"10.1177/25152459231156085","DOIUrl":"https://doi.org/10.1177/25152459231156085","url":null,"abstract":"Many psychological researchers use some form of a visual diagram in their research processes. Model diagrams used with structural equation models (SEMs) and causal directed acyclic graphs (DAGs) can guide causal-inference research. SEM diagrams and DAGs share visual similarities, often leading researchers familiar with one to wonder how the other differs. This article is intended to serve as a guide for researchers in the psychological sciences and psychiatric epidemiology on the distinctions between these methods. We offer high-level overviews of SEMs and causal DAGs using a guiding example. We then compare and contrast the two methodologies and describe when each would be used. In brief, SEM diagrams are both a conceptual and statistical tool in which a model is drawn and then tested, whereas causal DAGs are exclusively conceptual tools used to help guide researchers in developing an analytic strategy and interpreting results. Causal DAGs are explicitly tools for causal inference, whereas the results of a SEM are only sometimes interpreted causally. A DAG may be thought of as a “qualitative schematic” for some SEMs, whereas SEMs may be thought of as an “algebraic system” for a causal DAG. As psychology begins to adopt more causal-modeling concepts and psychiatric epidemiology begins to adopt more latent-variable concepts, the ability of researchers to understand and possibly combine both of these tools is valuable. Using an applied example, we provide sample analyses, code, and write-ups for both SEM and causal DAG approaches.","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":" ","pages":""},"PeriodicalIF":13.6,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42921040","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-04-01DOI: 10.1177/25152459231160105
S. Weston, Ian Shryock, Ryan Light, Phillip A. Fisher
Topic modeling is a type of text analysis that identifies clusters of co-occurring words, or latent topics. A challenging step of topic modeling is determining the number of topics to extract. This tutorial describes tools researchers can use to identify the number and labels of topics in topic modeling. First, we outline the procedure for narrowing down a large range of models to a select number of candidate models. This procedure involves comparing the large set on fit metrics, including exclusivity, residuals, variational lower bound, and semantic coherence. Next, we describe the comparison of a small number of models using project goals as a guide and information about topic representative and solution congruence. Finally, we describe tools for labeling topics, including frequent and exclusive words, key examples, and correlations among topics.
{"title":"Selecting the Number and Labels of Topics in Topic Modeling: A Tutorial","authors":"S. Weston, Ian Shryock, Ryan Light, Phillip A. Fisher","doi":"10.1177/25152459231160105","DOIUrl":"https://doi.org/10.1177/25152459231160105","url":null,"abstract":"Topic modeling is a type of text analysis that identifies clusters of co-occurring words, or latent topics. A challenging step of topic modeling is determining the number of topics to extract. This tutorial describes tools researchers can use to identify the number and labels of topics in topic modeling. First, we outline the procedure for narrowing down a large range of models to a select number of candidate models. This procedure involves comparing the large set on fit metrics, including exclusivity, residuals, variational lower bound, and semantic coherence. Next, we describe the comparison of a small number of models using project goals as a guide and information about topic representative and solution congruence. Finally, we describe tools for labeling topics, including frequent and exclusive words, key examples, and correlations among topics.","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":" ","pages":""},"PeriodicalIF":13.6,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42107656","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-04-01DOI: 10.1177/25152459221140388
Aaron J. Moss, David J. Hauser, Cheskie Rosenzweig, Shalom N Jaffe, Jonathan Robinson, L. Litman
Behavioral scientists looking to run online studies are confronted with a bevy of options. Where to recruit participants? Which tools to use for survey creation and study management? How to maintain data quality? In this tutorial, we highlight the unique capabilities of market-research panels and demonstrate how researchers can effectively sample from such panels. Unlike the microtask platforms most academics are familiar with (e.g., MTurk and Prolific), market-research panels have access to more than 100 million potential participants worldwide, provide more representative samples, and excel at demographic targeting. However, efficiently gathering data from online panels requires integration between the panel and a researcher’s survey in ways that are uncommon on microtask sites. For example, panels allow researchers to target participants according to preprofiled demographics (“Level 1” targeting, e.g., parents) and demographics that are not preprofiled but are screened for within the survey (“Level 2” targeting, e.g., parents of autistic children). In this article, we demonstrate how to sample hard-to-reach groups using market-research panels. We also describe several best practices for conducting research using online panels, including setting in-survey quotas to control sample composition and managing data quality. Our aim is to provide researchers with enough information to determine whether market-research panels are right for their research and to outline the necessary considerations for using such panels.
{"title":"Using Market-Research Panels for Behavioral Science: An Overview and Tutorial","authors":"Aaron J. Moss, David J. Hauser, Cheskie Rosenzweig, Shalom N Jaffe, Jonathan Robinson, L. Litman","doi":"10.1177/25152459221140388","DOIUrl":"https://doi.org/10.1177/25152459221140388","url":null,"abstract":"Behavioral scientists looking to run online studies are confronted with a bevy of options. Where to recruit participants? Which tools to use for survey creation and study management? How to maintain data quality? In this tutorial, we highlight the unique capabilities of market-research panels and demonstrate how researchers can effectively sample from such panels. Unlike the microtask platforms most academics are familiar with (e.g., MTurk and Prolific), market-research panels have access to more than 100 million potential participants worldwide, provide more representative samples, and excel at demographic targeting. However, efficiently gathering data from online panels requires integration between the panel and a researcher’s survey in ways that are uncommon on microtask sites. For example, panels allow researchers to target participants according to preprofiled demographics (“Level 1” targeting, e.g., parents) and demographics that are not preprofiled but are screened for within the survey (“Level 2” targeting, e.g., parents of autistic children). In this article, we demonstrate how to sample hard-to-reach groups using market-research panels. We also describe several best practices for conducting research using online panels, including setting in-survey quotas to control sample composition and managing data quality. Our aim is to provide researchers with enough information to determine whether market-research panels are right for their research and to outline the necessary considerations for using such panels.","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":"6 1","pages":""},"PeriodicalIF":13.6,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41396512","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-04-01DOI: 10.1177/25152459221151151
Zhicheng Lin, Qimin Ma, Yang Zhang
Studies in vision, psychology, and neuroscience often present visual stimuli on digital screens. Crucially, the appearance of visual stimuli depends on properties such as luminance and color, making it critical to measure them. Yet conventional luminance-measuring equipment is not only expensive but also onerous to operate (particularly for novices). Building on previous work, here we present an open-source integrated software package—PsyCalibrator (https://github.com/yangzhangpsy/PsyCalibrator)—that takes advantage of consumer hardware (SpyderX, Spyder5) and makes luminance/color measurement and gamma calibration accessible and flexible. Gamma calibration based on visual methods (without photometers) is also implemented. PsyCalibrator requires MATLAB (or its free alternative, GNU Octave) and works in Windows, macOS, and Linux. We first validated measurements from SpyderX and Spyder5 by comparing them with professional, high-cost photometers (ColorCAL MKII Colorimeter and Photo Research PR-670 SpectraScan). Validation results show (a) excellent accuracy in linear correction and luminance/color measurement and (b) for practical purposes, low measurement variances. We offer a detailed tutorial on using PsyCalibrator to measure luminance/color and calibrate displays. Finally, we recommend reporting templates to describe simple (e.g., computer-generated shapes) and complex (e.g., naturalistic images and videos) visual stimuli.
{"title":"PsyCalibrator: An Open-Source Package for Display Gamma Calibration and Luminance and Color Measurement","authors":"Zhicheng Lin, Qimin Ma, Yang Zhang","doi":"10.1177/25152459221151151","DOIUrl":"https://doi.org/10.1177/25152459221151151","url":null,"abstract":"Studies in vision, psychology, and neuroscience often present visual stimuli on digital screens. Crucially, the appearance of visual stimuli depends on properties such as luminance and color, making it critical to measure them. Yet conventional luminance-measuring equipment is not only expensive but also onerous to operate (particularly for novices). Building on previous work, here we present an open-source integrated software package—PsyCalibrator (https://github.com/yangzhangpsy/PsyCalibrator)—that takes advantage of consumer hardware (SpyderX, Spyder5) and makes luminance/color measurement and gamma calibration accessible and flexible. Gamma calibration based on visual methods (without photometers) is also implemented. PsyCalibrator requires MATLAB (or its free alternative, GNU Octave) and works in Windows, macOS, and Linux. We first validated measurements from SpyderX and Spyder5 by comparing them with professional, high-cost photometers (ColorCAL MKII Colorimeter and Photo Research PR-670 SpectraScan). Validation results show (a) excellent accuracy in linear correction and luminance/color measurement and (b) for practical purposes, low measurement variances. We offer a detailed tutorial on using PsyCalibrator to measure luminance/color and calibrate displays. Finally, we recommend reporting templates to describe simple (e.g., computer-generated shapes) and complex (e.g., naturalistic images and videos) visual stimuli.","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":" ","pages":""},"PeriodicalIF":13.6,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43447182","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-04-01DOI: 10.1177/25152459231168024
D. van den Bergh, E. Wagenmakers, F. Aust
Analysis of variance (ANOVA) is widely used to assess the influence of one or more experimental (or quasi-experimental) manipulations on a continuous outcome. Traditionally, ANOVA is carried out in a frequentist manner using p values, but a Bayesian alternative has been proposed. Assuming that the proposed Bayesian ANOVA is closely modeled after its frequentist counterpart, one may be surprised to find that the two can yield very different conclusions when the design involves multiple repeated-measures factors. We illustrate such a discrepancy with a real data set from a two-factorial within-subject experiment. For this data set, the results of a frequentist and Bayesian ANOVA are in a disagreement about which main effect accounts for the variance in the data. The reason for this disagreement is that frequentist and the proposed Bayesian ANOVA use different model specifications. As currently implemented, the proposed Bayesian ANOVA assumes that there are no individual differences in the magnitude of effects. We suspect that this assumption is neither obvious to nor desired by most analysts because it is untenable in most applications. We argue here that the Bayesian ANOVA should be revised to allow for individual differences. As a default, we suggest the standard frequentist model specification but discuss a recently proposed alternative and provide guidance on how to choose the appropriate model specification. We end by discussing the implications of the revised model specification for previously published results of Bayesian ANOVAs.
{"title":"Bayesian Repeated-Measures Analysis of Variance: An Updated Methodology Implemented in JASP","authors":"D. van den Bergh, E. Wagenmakers, F. Aust","doi":"10.1177/25152459231168024","DOIUrl":"https://doi.org/10.1177/25152459231168024","url":null,"abstract":"Analysis of variance (ANOVA) is widely used to assess the influence of one or more experimental (or quasi-experimental) manipulations on a continuous outcome. Traditionally, ANOVA is carried out in a frequentist manner using p values, but a Bayesian alternative has been proposed. Assuming that the proposed Bayesian ANOVA is closely modeled after its frequentist counterpart, one may be surprised to find that the two can yield very different conclusions when the design involves multiple repeated-measures factors. We illustrate such a discrepancy with a real data set from a two-factorial within-subject experiment. For this data set, the results of a frequentist and Bayesian ANOVA are in a disagreement about which main effect accounts for the variance in the data. The reason for this disagreement is that frequentist and the proposed Bayesian ANOVA use different model specifications. As currently implemented, the proposed Bayesian ANOVA assumes that there are no individual differences in the magnitude of effects. We suspect that this assumption is neither obvious to nor desired by most analysts because it is untenable in most applications. We argue here that the Bayesian ANOVA should be revised to allow for individual differences. As a default, we suggest the standard frequentist model specification but discuss a recently proposed alternative and provide guidance on how to choose the appropriate model specification. We end by discussing the implications of the revised model specification for previously published results of Bayesian ANOVAs.","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":" ","pages":""},"PeriodicalIF":13.6,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41864814","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-04-01DOI: 10.1177/25152459231156606
J. Fossum, A. Montoya
Several options exist for conducting inference on indirect effects in mediation analysis. Although methods that use bootstrapping are the preferred inferential approach for testing mediation, they are time-consuming when the test must be performed many times for a power analysis. Alternatives that are more computationally efficient are not as robust, meaning accuracy of the inferences from these methods is more affected by nonnormal and heteroskedastic data. Previous research has shown that different sample sizes are needed to achieve the same amount of statistical power for different inferential approaches with data that meet all the statistical assumptions of linear regression. By contrast, we explore how similar power estimates are at the same sample size, including when assumptions are violated. We compare the power estimates from six inferential methods for between-subjects mediation using a Monte Carlo simulation study. We varied the path coefficients, inferential methods for the indirect effect, and degree to which assumptions are met. We found that when the assumptions of linear regression are met, three inferential methods consistently perform similarly: the joint significance test, the Monte Carlo confidence interval, and the percentile bootstrap confidence interval. When the assumptions were violated, the nonbootstrapping methods tended to have vastly different power estimates compared with the bootstrapping methods. On the basis of these results, we recommend using the more computationally efficient joint significance test for power analysis only when no assumption violations are hypothesized a priori. We also recommend the joint significance test to pick an optimal starting sample size value for power analysis using the percentile bootstrap confidence interval when assumption violations are suspected.
{"title":"When to Use Different Inferential Methods for Power Analysis and Data Analysis for Between-Subjects Mediation","authors":"J. Fossum, A. Montoya","doi":"10.1177/25152459231156606","DOIUrl":"https://doi.org/10.1177/25152459231156606","url":null,"abstract":"Several options exist for conducting inference on indirect effects in mediation analysis. Although methods that use bootstrapping are the preferred inferential approach for testing mediation, they are time-consuming when the test must be performed many times for a power analysis. Alternatives that are more computationally efficient are not as robust, meaning accuracy of the inferences from these methods is more affected by nonnormal and heteroskedastic data. Previous research has shown that different sample sizes are needed to achieve the same amount of statistical power for different inferential approaches with data that meet all the statistical assumptions of linear regression. By contrast, we explore how similar power estimates are at the same sample size, including when assumptions are violated. We compare the power estimates from six inferential methods for between-subjects mediation using a Monte Carlo simulation study. We varied the path coefficients, inferential methods for the indirect effect, and degree to which assumptions are met. We found that when the assumptions of linear regression are met, three inferential methods consistently perform similarly: the joint significance test, the Monte Carlo confidence interval, and the percentile bootstrap confidence interval. When the assumptions were violated, the nonbootstrapping methods tended to have vastly different power estimates compared with the bootstrapping methods. On the basis of these results, we recommend using the more computationally efficient joint significance test for power analysis only when no assumption violations are hypothesized a priori. We also recommend the joint significance test to pick an optimal starting sample size value for power analysis using the percentile bootstrap confidence interval when assumption violations are suspected.","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":" ","pages":""},"PeriodicalIF":13.6,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48077177","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-01-01DOI: 10.1177/25152459221144986
Haiyang Jin, Qing Wang, Yufei Yang, Han Zhang, M. Gao, Shuxian Jin, Yanxiu (Sharon) Chen, Ting Xu, Yuan-Rui Zheng, Ji Chen, Q. Xiao, Jinbiao Yang, Xindi Wang, Haiyang Geng, Jianqiao Ge, Wei-Wei Wang, X. Chen, Lei Zhang, Xianli Zuo, H. Chuan-Peng
Open Science is becoming a mainstream scientific ideology in psychology and related fields. However, researchers, especially early-career researchers (ECRs) in developing countries, are facing significant hurdles in engaging in Open Science and moving it forward. In China, various societal and cultural factors discourage ECRs from participating in Open Science, such as the lack of dedicated communication channels and the norm of modesty. To make the voice of Open Science heard by Chinese-speaking ECRs and scholars at large, the Chinese Open Science Network (COSN) was initiated in 2016. With its core values being grassroots-oriented, diversity, and inclusivity, COSN has grown from a small Open Science interest group to a recognized network both in the Chinese-speaking research community and the international Open Science community. So far, COSN has organized three in-person workshops, 12 tutorials, 48 talks, and 55 journal club sessions and translated 15 Open Science-related articles and blogs from English to Chinese. Currently, the main social media account of COSN (i.e., the WeChat Official Account) has more than 23,000 subscribers, and more than 1,000 researchers/students actively participate in the discussions on Open Science. In this article, we share our experience in building such a network to encourage ECRs in developing countries to start their own Open Science initiatives and engage in the global Open Science movement. We foresee great collaborative efforts of COSN together with all other local and international networks to further accelerate the Open Science movement.
{"title":"The Chinese Open Science Network (COSN): Building an Open Science Community From Scratch","authors":"Haiyang Jin, Qing Wang, Yufei Yang, Han Zhang, M. Gao, Shuxian Jin, Yanxiu (Sharon) Chen, Ting Xu, Yuan-Rui Zheng, Ji Chen, Q. Xiao, Jinbiao Yang, Xindi Wang, Haiyang Geng, Jianqiao Ge, Wei-Wei Wang, X. Chen, Lei Zhang, Xianli Zuo, H. Chuan-Peng","doi":"10.1177/25152459221144986","DOIUrl":"https://doi.org/10.1177/25152459221144986","url":null,"abstract":"Open Science is becoming a mainstream scientific ideology in psychology and related fields. However, researchers, especially early-career researchers (ECRs) in developing countries, are facing significant hurdles in engaging in Open Science and moving it forward. In China, various societal and cultural factors discourage ECRs from participating in Open Science, such as the lack of dedicated communication channels and the norm of modesty. To make the voice of Open Science heard by Chinese-speaking ECRs and scholars at large, the Chinese Open Science Network (COSN) was initiated in 2016. With its core values being grassroots-oriented, diversity, and inclusivity, COSN has grown from a small Open Science interest group to a recognized network both in the Chinese-speaking research community and the international Open Science community. So far, COSN has organized three in-person workshops, 12 tutorials, 48 talks, and 55 journal club sessions and translated 15 Open Science-related articles and blogs from English to Chinese. Currently, the main social media account of COSN (i.e., the WeChat Official Account) has more than 23,000 subscribers, and more than 1,000 researchers/students actively participate in the discussions on Open Science. In this article, we share our experience in building such a network to encourage ECRs in developing countries to start their own Open Science initiatives and engage in the global Open Science movement. We foresee great collaborative efforts of COSN together with all other local and international networks to further accelerate the Open Science movement.","PeriodicalId":55645,"journal":{"name":"Advances in Methods and Practices in Psychological Science","volume":" ","pages":""},"PeriodicalIF":13.6,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45736253","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号