From worker empowerment to managerial control: The devolution of AI tools' intended positive implementation to their negative consequences

Abstract

AI systems are increasingly being deployed in organizations to support customer service (CS) departments. However, in practice, the introduction of AI tools often fails to meet these expectations and results in negative consequences, such as worker resistance and dissatisfaction. Yet we have little understanding of the process of how and why initially positive design intentions of AI tools result in negative consequences. Building on a qualitative in-depth case study of a Chinese firm introducing an AI tool in sales, we found that whereas the AI tool's initial design seemingly intended to lead to salespeople's empowerment and first achieved respective outcomes, over time the tool was appropriated for managerial control. We show that this devolution emerged organically from a growing managerial awareness of the affordances that the AI tool offered managers to perform their work better. Our study contributes to the literature on AI by highlighting the potential dangers of AI tools and emphasizing the importance of including workers in the AI tool's design and implementation phases.