Experimental evidence for rule learning as the underlying source of the item-position effect in reasoning ability measures

Abstract

For adequate description of reasoning test data, the consideration of the item-position effect (IPE) as a second latent variable in addition to reasoning ability is often required. The present study investigated the assumption that the learning of rules underlies the IPE. The factorial structure of two figural analogies tests was compared. 429 participants (age: 18–56 years) were randomly assigned to two conditions. In the continuous rule condition, the same rule had to be applied to all items and a typical IPE emerged. In the discontinuous rule condition, rules suddenly changed for the last items. This change led to the disruption of the IPE. A third latent variable was required to describe variance in the last items. Thus, the repetition of rules seems to be a precondition for a continuous IPE across test items. This is first evidence beyond correlations that individual differences in rule learning underlie the IPE.

Educational relevance

Reasoning tests are frequently used as an indicator of (general) intelligence and are valuable predictors of academic achievement. Our online experiment provides evidence for the notion that individual differences in reasoning tests are influenced by ad hoc rule learning during test taking that can be described as a latent variable and separated from the innate reasoning ability. These findings highlight the importance of not only looking at the overall performance but paying more attention to the dynamics of the test taking process itself.