From assistance to reliance: Development and validation of the large language model dependence scale

Abstract

With the rapid advancement of large language models (LLMs), the phenomenon of LLMs dependence has emerged and garnered significant attention. However, previous scales have been insufficient to measure the extent of individuals' dependence on LLMs. The current study aims to utilize the human-computer trust model and addiction theory to develop and validate the LLMs dependence scale (LDS) and to report its psychometric properties. An exploratory structural investigation of LLMs dependence was conducted with a sample of 421 LLMs users (Sample 1), which included items analysis, exploratory factor analysis, and network analysis. Additionally, a formal test was performed with a separate sample of 1030 LLMs users (Sample 2), with the data undergoing structural validation through confirmatory factor analysis, validity testing, and reliability testing. To mitigate the potential negative impacts of LLMs dependence, we employed the NodeIdentifyR algorithm for computational simulation interventions to identify critical intervention nodes within the LLMs dependence network. The results indicated that the LDS (18 items) exhibited a bifactor structure of functional dependence and existential dependence. The confirmatory factor model demonstrated a good fit and the LDS also showed good criterion-related validity. Subsequent simulated results of alleviating interventions suggested that users' existential dependence was primarily driven by their dependence on LLMs to handle tedious and boring tasks, while functional dependence was more influenced by users' belief in the omnipotence of LLMs. In summary, the factor structure of the LDS is clear, and its reliability and validity indices meet psychometric standards, making it an effective tool for measuring LLMs dependence.