Curse of Optimality, and How We Break It

Abstract

We strive to seek optimality, but often find ourselves trapped in bad "optimal" solutions that are either local optimizers, or too rigid to leave any room for errors, or simply based on wrong models. A way to break this "curse of optimality" is to engage exploration through randomization. Exploration broadens search space, provides flexibility, and facilitates learning via trial and error. We review some of the latest development of this exploratory approach in the stochastic control setting with continuous time and spaces.