Consumers' questions as nudges: Comparing the effect of linguistic cues on LLM chatbot and human responses

Abstract

Large Language Models (LLMs) enable natural language interactions, offering much potential for personalized consumer engagement in e-commerce. While prior studies have explored how chatbot expressions influence consumers, they often overlook the role of consumers as communicators who shape interactions through strategic language use. Additionally, evidence suggests that question cues can nudge humans to respond differently, but whether LLM chatbots adapt similarly to these cues remains underexplored. Informed by nudging theory, this study proposes that consumers can strategically use question cues to nudge LLM chatbots in response generation. Through a semantic analysis of 5676 responses to 1419 consumers' questions, we investigate the effects of cognitive and socio-emotional question cues on the informational and socio-emotional responses from humans and three LLM chatbots: ChatGPT, Claude and GLM. Findings suggest that, despite differing processing mechanisms, the nudging effect of question cues on LLM chatbots is similar as that on SNS users. Unlike humans, LLM chatbots exhibit a more pronounced tendency to focus exclusively on either informational or socio-emotional responses, rarely combining both aspects as seamlessly as humans do. This research underscores the importance of question formation in shaping consumer-chatbot interactions, suggesting that chatbots outperform humans in providing consistent informational responses regardless of question specificity and emotionality, while SNS users handle complex emotional queries more adeptly, showing their complementary roles in e-commerce.