Developing and Improving Personality Inventories Using Generative Artificial Intelligence: The Psychometric Properties of a Short HEXACO Scale Developed Using ChatGPT 4.0.

Abstract

In the current study, we investigated the utility of generative AI for survey development and improvement. To do so, we generated a 24-item HEXACO personality inventory using ChatGPT 4.0, the ChatGPT HEXACO inventory (CHI), and investigated whether ChatGPT could modify the CHI to either improve its internal consistency or its content validity. Additionally, we compared the psychometric properties of the different versions of the CHI to a conceptually similar short personality inventory. Specifically, we compared the three CHI versions with the Brief HEXACO inventory (BHI) in terms of their alpha reliabilities and their convergent and discriminant correlations with the HEXACO-60 and criterion-related validity with authoritarianism and social dominance orientation. Participants (N = 682) completed the BHI and HEXACO-60 and were randomly assigned to complete one of the three CHI versions. The results showed generally comparable psychometric properties of the three CHI versions and the BHI. However, ChatGPT could not improve specific psychometric properties of the CHI. That is, although the results show promise for the use of ChatGPT in developing questionnaires, it may not offer a shortcut to further improve specific psychometric properties.