Machine learning advice in managerial decision-making: The overlooked role of decision makers’ advice utilization

Machine learning (ML) analyses offer great potential to craft profound advice for augmenting managerial decision-making. Yet, even the most promising ML advice cannot improve decision-making if it is not utilized by decision makers. We therefore investigate how ML analyses influence decision makers’ utilization of advice and resulting decision-making performance. By analyzing data from 239 ML-supported decisions in real-world organizational scenarios, we demonstrate that decision makers’ utilization of ML advice depends on the information quality and transparency of ML advice as well as decision makers’ trust in data scientists’ competence. Furthermore, we find that decision makers’ utilization of ML advice can lead to improved decision-making performance, which is, however, moderated by the decision makers’ management level. The study’s results can help organizations leverage ML advice to improve decision-making and promote the mutual consideration of technical and social aspects behind ML advice in research and practice as a basic requirement.