Beyond effective use: Integrating wise reasoning in machine learning development

Abstract

The introduction of machine learning (ML), as the engine of many artificial intelligence (AI)-enabled systems in organizations, comes with the claim that ML models provide automated decisions or help domain experts improve their decision-making. Such a claim gives rise to the need to keep domain experts in the loop. Hence, data scientists, as those who develop ML models and infuse them with human intelligence during ML development, interact with various ML stakeholders and reflect their views within ML models. This interaction comes with (often conflicting) demands from various ML stakeholders and potential tensions. Building on the theories of effective use and wise reasoning, this mixed method study proposes a model to better understand how data scientists can use wisdom for managing these tensions when they develop ML models. In Study 1, through interviewing 41 analytics and ML experts, we investigate the dimensions of wise reasoning in the context of ML development. In Study 2, we test the overall model using a sample of 249 data scientists. Our results confirm that to develop effective ML models, data scientists need to not only use ML systems effectively, but also practice wise reasoning in their interactions with domain experts. We discuss the implications of these findings for research and practice.