Too much of a good thing: How varying levels of automation impact user performance in a simulated intrusion detection task

Abstract

Cyber analysts face a demanding task when prioritizing alerts from intrusion detection systems, balancing the challenge of numerous false positives from rule-based methods with the critical need to detect genuine cyber threats, necessitating unwavering vigilance and imposing a significant cognitive burden. In this field, there exists pressure to incorporate artificial intelligence techniques to enhance the automation of analyst workflows, yet without a clear grasp of how elevating the Level of Automation impacts the allocation of attentional and cognitive resources among analysts. This paper describes a simulated AI-assisted intrusion detection task which varies five degrees of automation as well as the sensitivity of the assistant, evaluating performance-based (e.g., accuracy, response time, sensitivity, response bias) and subjective (e.g., surveys on workload and trust) measures. Participants white-listed a series of time-sensitive alerts in a simulated Snort® environment. Our findings indicate that elevating the level of automation altered participants’ behavior, evident in their tendency to display a response bias towards rejecting hits (reduced hit rate and false alarm rate) when overriding an AI’s decision. Additionally, participants subjectively reported experiencing a decreased cognitive workload with a more precise algorithm, irrespective of any variance in their actual performance. Our findings suggest the necessity for additional research before implementing further automation into analyst workflows, as the demands of tasks evolve with escalating levels of automation.