Degradations of Trust in Automation Associated with Repeated Monitoring Checks

Abstract

Automated systems, or systems that execute, either partially or fully, a function previously completed by a human operator, have become widely incorporated in society today. As the use of automated systems becomes more prevalent, over-trust and subsequent over-reliance on those systems can occur and users can become complacent in monitoring. To ensure the user is correctly monitoring a system, many companies have incorporated monitoring checks to increase situational awareness (SA) and performance; however, there has not been much research evaluating the consequences of repeated monitoring checks. These checks could have consequences that negatively affect the user’s trust, such as the "cry-wolf" effect, because the user is repeatedly reminded of imperfections in the system resulting in disuse of the system. In contrast, these checks could help calibrate trust, increase SA and performance, and promote appropriate use of the system. Understanding these consequences is essential in evaluating a user’s trust in the system. To test this, an experiment was designed that explored the impact of monitoring checks and reliability on the level of trust individuals report in the system. In this study, the 60s and 90s frequency of monitoring check conditions had the highest reported trust. However, not all significant results were consistent with previous knowledge of the impacts of reliability and the "cry-wolf" effect. The general pattern of results suggests that there is merit in highly reliable systems especially over intermediately reliable systems. Additionally, the results suggest a complex relationship between trust and the frequency of monitoring checks especially at the intermediate frequency groups.