Nonlinear–nonquadratic optimal and inverse optimal control for discrete-time stochastic dynamical systems

Abstract

In this article, we investigate the role of Lyapunov functions in evaluating nonlinear–nonquadratic cost functionals for Itô-type nonlinear stochastic difference equations. Specifically, it is shown that the cost functional can be evaluated in closed-form as long as the cost functional is related in a specific way to an underlying Lyapunov function that guarantees asymptotic stability in probability. This result is then used to analyze discrete-time linear as well as nonlinear stochastic dynamical systems with polynomial and multilinear cost functionals. Furthermore, a stochastic optimal control framework is developed by exploiting connections between stochastic Lyapunov theory and stochastic Bellman theory. In particular, we show that asymptotic and geometric stability in probability of the closed-loop nonlinear system is guaranteed by means of a Lyapunov function that can clearly be seen to be the solution to the steady state form of the stochastic Bellman equation, and hence, guaranteeing both stochastic stability and optimality.