Semantic embeddings reveal and address taxonomic incommensurability in psychological measurement

Abstract

Taxonomic incommensurability denotes the difficulty in comparing scientific theories due to different uses of concepts and operationalizations. To tackle this problem in psychology, here we use language models to obtain semantic embeddings representing psychometric items, scales and construct labels in a vector space. This approach allows us to analyse different datasets (for example, the International Personality Item Pool) spanning thousands of items and hundreds of scales and constructs and show that embeddings can be used to predict empirical relations between measures, automatically detect taxonomic fallacies and suggest more parsimonious taxonomies. These findings suggest that semantic embeddings constitute a powerful tool for tackling taxonomic incommensurability in the psychological sciences. Taxonomic incommensurability highlights the difficulty of comparing scientific theories due to differing concepts and methods. This study uses language models to create semantic embeddings of psychometric items and scales, aiding in predicting empirical relations and clarifying psychological taxonomies.