Human Designers' Dynamic Confidence and Decision-Making When Working with More than One AI

Abstract

As artificial intelligence (AI) systems become increasingly capable of performing design tasks, they are expected to be deployed to assist human designers' decision-making in a greater variety of ways. For complex design problems such as those with multiple objectives, one AI may not always perform its expected accuracy due to the complexity of decision-making, and therefore multiples of AIs may be implemented to provide design suggestions. For such assistance to be productive, human designers must develop appropriate confidence in each AI and in themselves and accept or reject AI inputs accordingly. This work conducts a human subjects experiment to examine the development of a human designer's confidence in each AI and self-confidence throughout decision-making assisted by two AIs and how these confidences influence the decision to accept AI inputs. Major findings demonstrate that certain performance combinations of the two AIs and feedback lead to severe decreases in a human designer's confidences. Additionally, a human designer's decision to accept AI suggestions depends on their self-confidence and confidence in one of the two AIs. Finally, an additional AI does not increase a human designer's likelihood of conforming to AI suggestions. Therefore, in comparison to a scenario with one AI, the results in this work caution the implementation of an additional AI to AI-assisted decision-making scenarios. The insights also inform the design and management of human-AI teams to improve the outcome of AI-assisted decision-making.