Behavior Research Methods最新文献

Bayesian inference suggests that perception is inferred from a weighted integration of prior contextual beliefs with current sensory evidence (likelihood) about the world around us. The perceived precision or uncertainty associated with prior and likelihood information is used to guide perceptual decision-making, such that more weight is placed on the source of information with greater precision. This provides a framework for understanding a spectrum of clinical transdiagnostic symptoms associated with aberrant perception, as well as individual differences in the general population. While behavioral paradigms are commonly used to characterize individual differences in perception as a stable characteristic, measurement reliability in these behavioral tasks is rarely assessed. To remedy this gap, we empirically evaluate the reliability of a perceptual decision-making task that quantifies individual differences in Bayesian belief updating in terms of the relative precision weighting afforded to prior and likelihood information (i.e., sensory weight). We analyzed data from participants (n = 37) who performed this task twice. We found that the precision afforded to prior and likelihood information showed high internal consistency and good test-retest reliability (ICC = 0.73, 95% CI [0.53, 0.85]) when averaged across participants, as well as at the individual level using hierarchical modeling. Our results provide support for the assumption that Bayesian belief updating operates as a stable characteristic in perceptual decision-making. We discuss the utility and applicability of reliable perceptual decision-making paradigms as a measure of individual differences in the general population, as well as a diagnostic tool in psychiatric research.

Stimuli predicting rewards are more likely to capture attention, even when they are not relevant to our current goals. Individual differences in value-modulated attentional capture (VMAC) have been associated with various psychopathological conditions in the scientific literature. However, the claim that this attentional bias can predict individual differences requires further exploration of the psychometric properties of the most common experimental paradigms. The current study replicated the VMAC effect in a large online sample (N = 182) and investigated the internal consistency, with a design that allowed us to measure the effect during learning (rewarded phase) and after acquisition, once feedback was omitted (unrewarded phase). Through the rewarded phase there was gradual increase of the VMAC effect, which did not decline significantly throughout the unrewarded phase. Furthermore, we conducted a reliability multiverse analysis for 288 different data preprocessing specifications across both phases. Specifications including more blocks in the analysis led to better reliability estimates in both phases, while specifications that removed more outliers also improved reliability, suggesting that specifications with more, but less noisy, trials led to better reliability estimates. Nevertheless, in most instances, especially those considering fewer blocks of trials, reliability estimates fell below the minimum recommended thresholds for research on individual differences. Given the present results, we encourage researchers working on VMAC to take into account reliability when designing studies aimed at capturing individual differences and provide recommendations to improve methodological practices.

Multilevel modeling (MLM) is commonly used in psychological research to model clustered data. However, data in applied research usually violate one of the essential assumptions of MLM-homogeneity of variance. While the fixed-effect estimates produced by the maximum likelihood method remain unbiased, the standard errors for the fixed effects are misestimated, resulting in inaccurate inferences and inflated or deflated type I error rates. To correct the bias in fixed effects standard errors and provide valid inferences, small-sample corrections such as the Kenward-Roger (KR) adjustment and the adjusted cluster-robust standard errors (CR-SEs) with the Satterthwaite approximation for t tests have been used. The current study compares KR with random slope (RS) models and the adjusted CR-SEs with ordinary least squares (OLS), random intercept (RI) and RS models to analyze small, heteroscedastic, clustered data using a Monte Carlo simulation. Results show the KR procedure with RS models has large biases and inflated type I error rates for between-cluster effects in the presence of level 2 heteroscedasticity. In contrast, the adjusted CR-SEs generally yield results with acceptable biases and maintain type I error rates close to the nominal level for all examined models. Thus, when the interest is only in within-cluster effect, any model with the adjusted CR-SEs could be used. However, when the interest is to make accurate inferences of the between-cluster effect, researchers should use the adjusted CR-SEs with RS to have higher power and guard against unmodeled heterogeneity. We reanalyzed an example in Snijders & Bosker (2012) to demonstrate the use of the adjusted CR-SEs with different models.

This study established psycholinguistic norms in Cantonese for a set of 1286 colored pictures sourced from several picture databases, including 750 colored line drawings from MultiPic (Duñabeitia et al., 2018) and 536 photographs selected for McRae et al. (2005) concepts. The pictures underwent rigorous normalization processes. We provided picture characteristics including name and concept agreement, familiarity, visual complexity, and frequency of modal responses. Through correlational analyses, we observed strong interrelationships among these variables. We also compared the current Cantonese norming to other languages and demonstrated similarity and variations among different languages. Additionally, we embraced the multilingual diversity within the current sample, and found that higher Cantonese proficiency but lower non-native language proficiency were associated with better spoken picture naming. Last but not least, we validated the predictive power of normed variables calculated from typed responses to spoken picture naming, and the consistency between typed and spoken responses. The present norming provides a timely and valuable alternative for researchers in the field of psycholinguistics, especially those studying Cantonese production and lexical retrieval. All raw data, analysis scripts, and final norming results are available online as psycholinguistic norms for Cantonese in the following link at the Open Science Framework: https://osf.io/dz9j6/?view_only=a452d8a56c92430b9dedf21ac26b1bc1 .

Coefficient alpha is commonly used as a reliability estimator. However, several estimators are believed to be more accurate than alpha, with factor analysis (FA) estimators being the most commonly recommended. Furthermore, unstandardized estimators are considered more accurate than standardized estimators. In other words, the existing literature suggests that unstandardized FA estimators are the most accurate regardless of data characteristics. To test whether this conventional knowledge is appropriate, this study examines the accuracy of 12 estimators using a Monte Carlo simulation. The results show that several estimators are more accurate than alpha, including both FA and non-FA estimators. The most accurate on average is a standardized FA estimator. Unstandardized estimators (e.g., alpha) are less accurate on average than the corresponding standardized estimators (e.g., standardized alpha). However, the accuracy of estimators is affected to varying degrees by data characteristics (e.g., sample size, number of items, outliers). For example, standardized estimators are more accurate than unstandardized estimators with a small sample size and many outliers, and vice versa. The greatest lower bound is the most accurate when the number of items is 3 but severely overestimates reliability when the number of items is more than 3. In conclusion, estimators have their advantageous data characteristics, and no estimator is the most accurate for all data characteristics.

Virtual reality (VR) displays are being used in an increasingly wide range of applications. However, previous work shows that viewers often perceive scene properties very differently in real and virtual environments and so realistic perception of virtual stimuli should always be a carefully tested conclusion, not an assumption. One important property for realistic scene perception is surface color. To evaluate how well virtual platforms support realistic perception of achromatic surface color, we assessed lightness constancy in a physical apparatus with real lights and surfaces, in a commercial VR headset, and on a traditional flat-panel display. We found that lightness constancy was good in all three environments, though significantly better in the real environment than on the flat-panel display. We also found that variability across observers was significantly greater in VR and on the flat-panel display than in the physical environment. We conclude that these discrepancies should be taken into account in applications where realistic perception is critical but also that in many cases VR can be used as a flexible alternative to flat-panel displays and a reasonable proxy for real environments.

Preregistration has gained traction as one of the most promising solutions to improve the replicability of scientific effects. In this project, we compared 193 psychology studies that earned a Preregistration Challenge prize or preregistration badge to 193 related studies that were not preregistered. In contrast to our theoretical expectations and prior research, we did not find that preregistered studies had a lower proportion of positive results (Hypothesis 1), smaller effect sizes (Hypothesis 2), or fewer statistical errors (Hypothesis 3) than non-preregistered studies. Supporting our Hypotheses 4 and 5, we found that preregistered studies more often contained power analyses and typically had larger sample sizes than non-preregistered studies. Finally, concerns about the publishability and impact of preregistered studies seem unwarranted, as preregistered studies did not take longer to publish and scored better on several impact measures. Overall, our data indicate that preregistration has beneficial effects in the realm of statistical power and impact, but we did not find robust evidence that preregistration prevents p-hacking and HARKing (Hypothesizing After the Results are Known).

Most natural language models and tools are restricted to one language, typically English. For researchers in the behavioral sciences investigating languages other than English, and for those researchers who would like to make cross-linguistic comparisons, hardly any computational linguistic tools exist, particularly none for those researchers who lack deep computational linguistic knowledge or programming skills. Yet, for interdisciplinary researchers in a variety of fields, ranging from psycholinguistics, social psychology, cognitive psychology, education, to literary studies, there certainly is a need for such a cross-linguistic tool. In the current paper, we present Lingualyzer ( https://lingualyzer.com ), an easily accessible tool that analyzes text at three different text levels (sentence, paragraph, document), which includes 351 multidimensional linguistic measures that are available in 41 different languages. This paper gives an overview of Lingualyzer, categorizes its hundreds of measures, demonstrates how it distinguishes itself from other text quantification tools, explains how it can be used, and provides validations. Lingualyzer is freely accessible for scientific purposes using an intuitive and easy-to-use interface.

Reading is a key skill for university students. The Author Recognition Test (ART) and the Title Recognition Test (TRT) have both been used for decades to measure print exposure which correlates with reading and other linguistic skills. Given the available evidence for interindividual differences in reading skills, this study addresses three open issues. First, to what extent do ART and TRT scores correlate with individual differences regarding students' study programs? Second, how do these results correlate with the self-reported time spent reading different types of text genres (e.g., fiction, nonfiction) per week? And third, this study compares ART and TRT to one another. We tested students from six study programs in the humanities and (medical) sciences which vary in the amount and kind of reading material required for study success. We found that students perform significantly differently in the ART and the TRT depending on their field of study. Students in a study program focusing on fiction and literature perform best overall. We also replicated the well-known effect of age on ART and TRT scores: older students have better scores. We did not find reliable effects of reading time on test performance, whereas individual creative writing habits did positively predict ART/TRT test results. These results raise a number of important questions regarding the ART/TRT in general and regarding interindividual differences in personal reading and writing habits and the change in reading habits in times of media convergence.

Carelessness or insufficient effort responding is a widespread problem in online research, with estimates ranging from 3% to almost 50% of participants in online surveys being inattentive. While detecting carelessness has been subject to multiple studies, the factors that reduce or prevent carelessness are not as well understood. Initial evidence suggests that warning statements prior to study participation may reduce carelessness, but there is a lack of conclusive high-powered studies. This preregistered randomized controlled experiment aimed to test the effectiveness of a warning statement and an improved implementation of a warning statement in reducing participant inattention. A study with 812 participants recruited on Amazon Mechanical Turk was conducted. Results suggest that presenting a warning statement is not effective in reducing carelessness. However, requiring participants to actively type the warning statement statistically significantly reduced carelessness as measured with self-reported diligence, even-odd consistency, psychometric synonyms and antonyms, and individual response variability. The active warning statements also led to statistically significantly more attrition and potentially deterred those who were likely to be careless from even participating in this study. We show that the current standard practice of implementing warning statements is ineffective and novel methods to prevent and deter carelessness are needed.