Behavior Research Methods最新文献

Mixed-format tests, which typically include dichotomous items and polytomously scored tasks, are employed to assess a wider range of knowledge and skills. Recent behavioral and educational studies have highlighted their practical importance and methodological developments, particularly within the context of multivariate generalizability theory. However, the diverse response types and complex designs of these tests pose significant analytical challenges when modeling data simultaneously. Current methods often struggle to yield reliable results, either due to the inappropriate treatment of different types of response data separately or the imposition of identical covariates across various response types. Moreover, there are few software packages or programs that offer customized solutions for modeling mixed-format tests, addressing these limitations. This tutorial provides a detailed example of using a Bayesian approach to model data collected from a mixed-format test, comprising multiple-choice questions and free-response tasks. The modeling was conducted using the Stan software within the R programming system, with Stan codes tailored to the structure of the test design, following the principles of multivariate generalizability theory. By further examining the effects of prior distributions in this example, this study demonstrates how the adaptability of Bayesian models to diverse test formats, coupled with their potential for nuanced analysis, can significantly advance the field of psychometric modeling.

Episodic memory may essentially be memory for one's place within a temporally unfolding scene from a first-person perspective. Given this, pervasively used static stimuli may only capture one small part of episodic memory. A promising approach for advancing the study of episodic memory is immersing participants within varying scenes from a first-person perspective. We present a pool of distinct scene stimuli for use in virtual environments and a paradigm that is implementable across varying levels of immersion on multiple virtual reality (VR) platforms and adaptable to studying various aspects of scene and episodic memory. In our task, participants are placed within a series of virtual environments from a first-person perspective and guided through a virtual tour of scenes during a study phase and a test phase. In the test phase, some scenes share a spatial layout with studied scenes; others are completely novel. In three experiments with varying degrees of immersion, we measure scene recall, scene familiarity-detection during recall failure, the subjective experience of déjà vu, the ability to predict the next turn on a tour, the subjective sense of being able to predict the next turn on a tour, and the factors that influence memory search and the inclination to generate candidate recollective information. The level of first-person immersion mattered to multiple facets of episodic memory. The paradigm presents a useful means of advancing mechanistic understanding of how memory operates in realistic dynamic scene environments, including in combination with cognitive neuroscience methods such as functional magnetic resonance imaging and electrophysiology.

Parsing signals from noise is a general problem for signallers and recipients, and for researchers studying communicative systems. Substantial efforts have been invested in comparing how other species encode information and meaning, and how signalling is structured. However, research depends on identifying and discriminating signals that represent meaningful units of analysis. Early approaches to defining signal repertoires applied top-down approaches, classifying cases into predefined signal types. Recently, more labour-intensive methods have taken a bottom-up approach describing detailed features of each signal and clustering cases based on patterns of similarity in multi-dimensional feature-space that were previously undetectable. Nevertheless, it remains essential to assess whether the resulting repertoires are composed of relevant units from the perspective of the species using them, and redefining repertoires when additional data become available. In this paper we provide a framework that takes data from the largest set of wild chimpanzee (Pan troglodytes) gestures currently available, splitting gesture types at a fine scale based on modifying features of gesture expression using latent class analysis (a model-based cluster detection algorithm for categorical variables), and then determining whether this splitting process reduces uncertainty about the goal or community of the gesture. Our method allows different features of interest to be incorporated into the splitting process, providing substantial future flexibility across, for example, species, populations, and levels of signal granularity. Doing so, we provide a powerful tool allowing researchers interested in gestural communication to establish repertoires of relevant units for subsequent analyses within and between systems of communication.

Determining the compositional structure and dimensionality of psychological constructs lies at the heart of many research questions in developmental science. Structural equation modeling (SEM) provides a versatile framework for formalizing and estimating the relationships among multiple latent constructs. While the flexibility of SEM can accommodate many complex assumptions on the underlying structure of psychological constructs, it makes a priori estimation of statistical power and required sample size challenging. This difficulty is magnified when comparing non-nested SEMs, which prevents the use of traditional likelihood-ratio tests. Sample size estimates for SEM model fit comparisons typically rely on generic rules of thumb. Such heuristics can be misleading because statistical power in SEM depends on a variety of model properties. Here, we demonstrate a Monte Carlo simulation approach for estimating a priori statistical power for model selection when comparing non-nested models in an SEM framework. We provide a step-by-step guide to this approach based on an example from our memory development research in children.

A methodological problem in most reaction time (RT) studies is that some measured RTs may be outliers-that is, they may be very fast or very slow for reasons unconnected to the task-related processing of interest. Numerous ad hoc methods have been suggested to discriminate between such outliers and the valid RTs of interest, but it is extremely difficult to determine how well these methods work in practice because virtually nothing is known about the actual characteristics of outliers in real RT datasets. This article proposes a new method of pooling cumulative distribution function values for examining empirical RT distributions to assess both the proportions of outliers and their latencies relative to those of the valid RTs. As the method is developed, its strengths and weaknesses are examined using simulations based on previously suggested ad hoc models for RT outliers with particular assumed proportions and distributions of valid RTs and outliers. The method is then applied to several large RT datasets from lexical decision tasks, and the results provide the first empirically based description of outlier RTs. For these datasets, fewer than 1% of the RTs seem to be outliers, and the median outlier latency appears to be approximately 4-6 standard deviations of RT above the mean of the valid RT distribution.

The assessment of creativity as an individual difference has historically focused on divergent thinking, which is increasingly viewed as involving the associative processes that are also understood to be a key component of creative potential. Research on associative processes has proliferated in many sub-fields, often using Compound Remote Associates (CRA) tasks with an open response format and relatively small participant samples. In the present work, we introduce a new format that is more amenable to large-scale data collection in survey designs, and present evidence for the reliability and validity of CRA measures in general using multiple large samples. Study 1 uses a large, representative dataset (N = 1,323,480) to demonstrate strong unidimensionality and internal consistency (α = .97; ωt = .87), as well as links to individual differences in temperament, cognitive ability, occupation, and job characteristics. Study 2 uses an undergraduate sample (N = 685) to validate the use of a multiple-choice format relative to the traditional approach. Study 3 uses a crowdsourced sample (N = 357) to demonstrate high test-retest reliability of the items (r =.74). Finally, Study 4 uses a sample that overlaps with Study 1 (N = 1,502,922) to provide item response theory (IRT) parameters for a large set of high-quality CRA items that use a multiple-choice response mode, thus facilitating their use in future research on creativity, insight, and related topics.

There have been many published picture corpora. However, more than half of the world's population speaks more than one language and, as language and culture are intertwined, some of the items from a picture corpus designed for a given language in a particular culture may not fit another culture (with the same or different language). There is also an awareness that language research can gain from the study of bi-/multilingual individuals who are immersed in multilingual contexts that foster inter-language interactions. Consequently, we developed a relatively large corpus of pictures (663 nouns, 96 verbs) and collected normative data from multilingual speakers of Kannada (a southern Indian language) on two picture-related measures (name agreement, image agreement) and three word-related measures (familiarity, subjective frequency, age of acquisition), and report objective visual complexity and syllable count of the words. Naming labels were classified into words from the target language (i.e., Kannada), cognates (borrowed from/shared with another language), translation equivalents, and elaborations. The picture corpus had > 85% mean concept agreement with multiple acceptable names (1-7 naming labels) for each concept. The mean percentage name agreement for the modal name was > 70%, with H-statistics of 0.89 for nouns and 0.52 for verbs. We also analyse the variability of responses highlighting the influence of bi-/multilingualism on (picture) naming. The picture corpus is freely accessible to researchers and clinicians. It may be used for future standardization with other languages of similar cultural contexts, and relevant items can be used in languages from different cultures, following suitable standardization.

The discriminability measure $d^{\text{'}}$ is widely used in psychology to estimate sensitivity independently of response bias. The conventional approach to estimate $d^{\text{'}}$ involves a transformation from the hit rate and the false-alarm rate. When performance is perfect, correction methods must be applied to calculate $d^{\text{'}}$ , but these corrections distort the estimate. In three simulation studies, we show that distortion in $d^{\text{'}}$ estimation can arise from other properties of the experimental design (number of trials, sample size, sample variance, task difficulty) that, when combined with application of the correction method, make $d^{\text{'}}$ distortion in any specific experiment design complex and can mislead statistical inference in the worst cases (Type I and Type II errors). To address this problem, we propose that researchers simulate $d^{\text{'}}$ estimation to explore the impact of design choices, given anticipated or observed data. An R Shiny application is introduced that estimates $d^{\text{'}}$ distortion, providing researchers the means to identify distortion and take steps to minimize its impact.

This research is an extension of the recent scale development efforts for the marker variable Attitude Toward the Color Blue (ATCB), which addresses the efficacy of multiple shorter permutations of the scale. The purpose of this study is to develop a shorter version of an ideal marker variable scale used to detect common method variance. Potential uses of the shorter version of ATCB include intensive longitudinal studies, implementation of experience sampling methodology, or any brief survey for which the original version might be cumbersome to implement repeatedly or appear very odd to the respondent when paired with only a few other substantive items. Study 1, uses all six-, five-, and four-item versions of ATCB in confirmatory factor analysis (CFA) marker technique tests on a bivariate relationship. Study 2 analyzes the best- and worst-performing versions of reduced lengths of the ATCB scale found in the first study on another bivariate relationship. Study 3 compares the original seven-item version, as well as randomly selected reduced length versions in a data set with 15 model relationships. Study 4 uses an experiment to determine the efficacy of providing respondents with one of three shorter ATCB scales in a model of three substantive variables. Our findings indicate that ATCB of different permutations and lengths can detect CMV successfully, and that researchers should choose the length of scale based on their survey length. We conclude that ATCB is adaptable for a variety of research situations, presenting it as a valuable tool for high-quality research.

The tendency for individuals to mind wander is often measured using experience sampling methods in which probe questions embedded within computerized cognitive tasks attempt to catch episodes of off-task thought at random intervals during task performance. However, mind-wandering probe questions and response options are often chosen ad hoc and vary between studies with extant little guidance as to the psychometric consequences of these methodological decisions. In the present study, we examined the psychometric properties of several common approaches for assessing mind wandering using methods from item response theory (IRT). IRT latent modeling demonstrated that measurement information was generally distributed across the range of trait estimates according to when probes were presented in time. Probes presented earlier in time provided more information about individuals with greater tendency to mind wandering than probes presented later. Furthermore, mind-wandering ratings made on a continuous scale or using multiple categorical rating options provided more information about individuals' latent mind-wandering tendency - across a broader range of the trait continuum - than ratings dichotomized into on-task and off-task categories. In addition, IRT provided evidence that reports of "task-related thoughts" contribute to the task-focused dimension of the construct continuum, providing justification for studies conceptualizing these responses as a kind of task-related focus. Together, we hope these findings will help guide researchers hoping to maximize the measurement precision of their mind wandering assessment procedures.