Systems Level Evaluation of Resilience in Human-Autonomy Teaming under Degraded Conditions

Abstract

In this longitudinal study, we examined the performance of Human Autonomy Teams (HATs) in the context of a Remotely Piloted Aircraft System (RPAS) to determine team resilience of HATs under three types of degraded conditions – an automation failure, an autonomy failure, and a malicious cyber-attack. In this study, two human team members interacted with a “synthetic” agent who was actually a well-trained experimenter. First, we identified high- and low-performing teams by considering team performance score and overcoming number of failures across 10 40-minute missions. We calculated the amount of system level entropy (extracted from human and technological signals) over the course of the missions to track the amount of system reorganization in response to failures. We hypothesized that resilient teams would be more effective at reorganizing system level behavior, as observed through entropy. To explore team resilience, we examined how long it took these two teams to overcome the failures, as well as the amount of system reorganization each team displayed throughout the failure. Our findings from this exploratory analysis indicate that the high-performing team displayed more flexibility and adaptivity under degraded conditions than the low-performing team. This also underlines that effective systems level reorganization is needed in order to be adaptive and resilient in a dynamic task environment.