Behavior Research Methods最新文献

The measurement efficiency of a grid multidimensional computerized classification test (grid MCCT), which makes a classification decision per dimension, can be improved by taking the correlations between the dimensions into account in the termination criterion. The higher the correlations, the better the improvement in measurement efficiency. However, a termination criterion utilizing inter-dimensional information (i.e., SPRT-C; Liu et al., 2022) was found to yield lower levels of correct classification rates than not utilizing it (i.e., SPRT-SF; Seitz & Frey, 2013) under the between-item grid MCCT when the cutoff was set at the mean of the latent trait distribution. This study proposes a pre-rule to determine whether the SPRT-SF or SPRT-C should be used during the process of classification test administration. Through a series of simulation studies, the results showed that our proposed method (called P-SPRT) can substantially improve upon the SPRT-C in terms of correct classification rates, while maintaining its high measurement efficiency in terms of test length. This paper concludes with a discussion of the findings and further applications.

We introduce EmoAtlas, a computational library/framework extracting emotions and syntactic/semantic word associations from texts. EmoAtlas combines interpretable artificial intelligence (AI) for syntactic parsing in 18 languages and psychologically validated lexicons for detecting the eight emotions in Plutchik's theory. We show that EmoAtlas can match or surpass transformer-based natural language processing techniques, BERT or large language models like ChatGPT 3.5 or LLaMAntino, in detecting emotions from Italian and English online posts and news articles (e.g., achieving 85.6 $\%$ accuracy in detecting anger in posts vs the 68.8 $\%$ value of ChatGPT and 89.9% value for BERT). EmoAtlas presents important advantages in terms of speed and absence of fine-tuning, e.g., it runs 12x faster than BERT on the same data. Testing EmoAtlas' and easily trainable transformers' relevance in a psychometric task like reproducing human creativity ratings for 1071 short texts, we find that EmoAtlas and BERT obtain equivalent predictive power (fourfold cross-validation, $\rho \approx 0.495$ , $p<{10}^{-4}$ ). Combining BERT's semantic features with EmoAtlas' emotional/syntactic networks of words gets substantially better at estimating creativity rates of stories ( $\rho =0.628$ , $p<{10}^{-4}$ ). This indicates an interplay between the creativity of narratives and their semantic, emotional, and syntactic structure. Via interpretable emotional profiles and syntactic networks, EmoAtlas can also quantify how emotions are channeled through specific words in texts, e.g., how did customers frame their ideas and emotions towards "beds" in hotel reviews? We release EmoAtlas as a standalone "text as data" computational tool and discuss its impact in extracting interpretable and reproducible insights from texts.

The autobiographical implicit association test (aIAT) is an approach of memory detection that can be used to identify true autobiographical memories. This study incorporates mouse-tracking (MT) into aIAT, which offers a more robust technique of memory detection. Participants were assigned to mock crime and then performed the aIAT with MT. Results showed that mouse metrics exhibited IAT effects that correlated with the IAT effect of RT and showed differences in autobiographical and irrelevant events while RT did not. Our findings suggest the validity of MT in offering measurement of the IAT effect. We also observed different patterns in mouse trajectories and velocity for autobiographical and irrelevant events. Lastly, utilizing MT metric, we identified that the Past Negative Score was positively correlated with IAT effect. Integrating the Past Negative Score and AUC into computational models improved the simulation results. Our model captured the ubiquitous implicit association between autobiographical events and the attribute True, and offered a mechanistic account for implicit bias. Across the traditional IAT and the MT results, we provide evidence that MT-aIAT can better capture the memory identification and with implications in crime detection.

Delay discounting (DD) describes the tendency of individuals to devalue the worth of a reward as a function of the delay in receiving it. DD is impaired in many clinical conditions and changes across development. Many existing automated DD tasks are built on copyrighted software and primarily designed for English speakers, which hinders content editing and accessibility. Given this scenario, we had three objectives: (1) to develop open-source DD software named the "Waiting Game" with a user interface (UI) that is easily editable (regarding language, reward type/magnitude and delay duration) via an Excel spreadsheet, and provides automated DD scoring; (2) to create a comprehensive manual (User Guide) to accompany the software; and (3) to assess the software's usability and the clarity of the manual through an online questionnaire completed by experts in cognitive assessment. The software was developed using game design and encompasses three tasks that assess DD under three conditions: (1) hypothetical delays (waiting is imagined) and no real rewards (only points) are gained); (2) real delays (waiting is necessary) and real rewards gained; and (3) real delays and hypothetical rewards. An expert evaluation using the System Usability Scale and the International Test Commission recommendations confirmed the software's suitability. Minor changes were made to the User Guide and UI based on the expert feedback. We conclude that the Waiting Game offers a valid, cost-free, and automated solution for DD assessment that facilitates reward and delay manipulations in hypothetical/real delay and reward paradigms across diverse sociocultural contexts.

The classical meta-analytical random effects model (REM) has some weaknesses when applied to the standardized mean difference, g. Essentially, the variance of the studies involved is taken as the conditional variance, given a δ value, instead of the unconditional variance. As a consequence, the estimators of the variances involve a dependency between the g values and their variances that distorts the estimates. The classical REM is expressed as a linear model and the variance of g is obtained through a framework of components of variance. Although the weaknesses of the REM are negligible in practical terms in a wide range of realistic scenarios, all together, they make up an approximate, simplified version of the meta-analytical random effects model. We present an alternative formulation, as a mixture model, and provide formulas for the expected value, variance and skewness of the marginal distribution of g. A Monte Carlo simulation supports the accuracy of the formulas. Then, unbiased estimators of both the mean and the variance of the true effects are proposed, and assessed through Monte Carlo simulations. The advantages of the mixture model formulation over the "classical" formulation are discussed.

In this article, we discuss operationalizations and examples of experimental design in eye-tracking research. First, we distinguish direct operationalization for entities like saccades, which are closely aligned with their original concepts, and indirect operationalization for concepts not directly measurable, such as attention or mind-wandering. The latter relies on selecting a measurable proxy. Second, we highlight the variability in algorithmic operationalizations and emphasize that changing parameters can affect outcome measures. Transparency in reporting these parameters and algorithms is crucial for comparisons across studies. Third, we provide references to studies for common operationalizations in eye-tracking research and discuss key operationalizations in reading research. Fourth, the IO-model is introduced as a tool to help researchers operationalize difficult concepts. Finally, we present three example experiments with useful methods for eye-tracking research, encouraging readers to consider these examples for inspiration in their own experiments.

In reading tasks, drift can move fixations from one word to another or even another line, invalidating the eye-tracking recording. Manual correction is time-consuming and subjective, while automated correction is fast - yet limited in accuracy. In this paper, we present Fix8 (Fixate), an open-source GUI tool that offers a novel semi-automated correction approach for eye-tracking data in reading tasks. The proposed approach allows the user to collaborate with an algorithm to produce accurate corrections faster without sacrificing accuracy. Through a usability study (N = 14) we assess the time benefits of the proposed technique, and measure the correction accuracy in comparison to manual correction. In addition, we assess subjective workload through the NASA Task Load Index, and user opinions through Likert-scale questions. Our results show that, on average, the proposed technique was 44% faster than manual correction without any sacrifice of accuracy. In addition, users reported a preference for the proposed technique, lower workload, and higher perceived performance compared to manual correction. Fix8 is a valuable tool that offers useful features for generating synthetic eye-tracking data, visualization, filters, data converters, and eye-movement analysis in addition to the main contribution in data correction.

Due to the increased availability of intensive longitudinal data, researchers have been able to specify increasingly complex dynamic latent variable models. However, these models present challenges related to overfitting, hierarchical features, non-linearity, and sample size requirements. There are further limitations to be addressed regarding the finite sample performance of priors, including bias, accuracy, and type I error inflation. Bayesian estimation provides the flexibility to treat these issues simultaneously through the use of regularizing priors. In this paper, we aim to compare several Bayesian regularizing priors (ridge, Bayesian Lasso, adaptive spike-and-slab Lasso, and regularized horseshoe). To achieve this, we introduce a multilevel dynamic latent variable model. We then conduct two simulation studies and a prior sensitivity analysis using empirical data. The results show that the ridge prior is able to provide sparse estimation while avoiding overshrinkage of relevant signals, in comparison to other Bayesian regularization priors. In addition, we find that the Lasso and heavy-tailed regularizing priors do not perform well compared to light-tailed priors for the logistic model. In the context of multilevel dynamic latent variable modeling, it is often attractive to diversify the choice of priors. However, we instead suggest prioritizing the choice of ridge priors without extreme shrinkage, which we show can handle the trade-off between informativeness and generality, compared to other priors with high concentration around zero and/or heavy tails.

Eye tracking has been a popular methodology used to study the visual, cognitive, and linguistic processes underlying word recognition and sentence parsing during reading for several decades. However, the successful use of eye tracking requires researchers to make deliberate choices about how they apply this technique, and there is wide variability across labs and fields with respect to which choices are "standard." We aim to provide an easy-to-reference guideline that can help new researchers with their entrée into eye-tracking-while-reading research. Because the standards do - and should - vary from field to field or study to study as is appropriate for the research question, we do not set a rigid recipe for handling eye tracking data, but rather provide a conceptual framework within which researchers can make informed decisions about how to treat their data so that it is most informative for their research question. Therefore, this paper provides a description of eye movements in reading and an overview of psycholinguistic research on the topic, an overview of experiment design considerations, a description of the data processing pipeline and important choice points and implications, an overview of common dependent measures and their calculation, and a summary of resources for data analysis.