Performance–Robustness Tradeoffs in Adversarially Robust Control and Estimation

Abstract

Efforts by the reinforcement learning community to close the sim-to-real gap have resulted in policy optimization objectives, which are distinct from, although related to, existing objectives in robust control, such as ${\mathcal {H}}_{\infty }$ methods. The disparity from the familiar control methods makes it challenging to make rigorous claims about these methods, and to predict the implications on performance of training a policy with a particular level of robustness. This in turn makes selecting the level of robustness a heavily heuristic exercise. Toward addressing these issues, we study the synthesis problem for a control objective consisting of both zero-mean stochastic disturbances, and bounded adversarial disturbances entering the state and measurement under linear dynamics and quadratic cost. We show that this problem admits a linear time-invariant controller that has a form closely related to suboptimal ${\mathcal {H}}_{\infty }$ solutions. We also study the tradeoffs induced by optimizing the control objective in the presence of an adversary by examining how such a solution degrades controller performance in the absence of an adversary. To this end, we provide a quantitative performance–robustness tradeoff analysis in two analytically tractable cases: state feedback control and state prediction. In these special cases, we demonstrate that the severity of the tradeoff depends in an interpretable manner upon system-theoretic properties, such as the spectrum of the controllability Gramian, the spectrum of the observability Gramian, and the stability of the system. This may provide practitioners guidance for determining how much robustness to incorporate based on a priori system knowledge, and conversely how to design systems where the tradeoff is less severe. We empirically validate our results by comparing the performance of the controller against standard baselines, and plotting performance–robustness tradeoff curves.