Psychometric Modelling: A Systematic Critique of Underlying Assumptions and Some Alternatives

Abstract

Due to the popularity of psychometrics, where probabilistic modelling takes a leading position, it seems useful to present a critical view of these techniques, which, unfortunately, is not extensively discussed, especially in the Russian-language literature. This paper is a systematic review of the growing criticism of several assumptions of psychometrics: 1) the existence of unobservable personality traits and abilities which are supposed to determine the performance of test items; 2) the stochastic nature of traits, and as a consequence, the need for probabilistic models to study them; 3) aggregate data represent individual characteristics; 4) as in the natural sciences, models in psychometrics help reveal psychological reality that is not directly observable; 5) if testing the agreement of empirical data with a model yields a positive result, then the model correctly describes reality. These assumptions are criticized on the basis of the circular nature of the definitions of the traits themselves, logical errors ingrained in the assumptions, the methodological mixture of variability and randomness regarding behavior, the lack of a causal link between inter-individual variation in responses and individual responses; superficial analogies with models in the natural science to which psychometrics refers, as well as the substitution of scientific tasks for instrumental and pragmatic ones. We conclude that modelling in psychometrics is counterproductive if used as a method of exploring the psychological reality behind the test. Some alternative practices of quantitative research are discussed. For example, testing the existence of variation at the level of the individual and the experimental search for its explanation. There are other possible alternatives, such as, a network perspective on psychological phenomena; faceted theory; or observation-oriented modeling. Although this kind of research is much more difficult to implement than standard “goodness-of-fit” tests, it is probably this kind of research that can provide an increase in psychological knowledge.