Time Consistency for Multistage Stochastic Optimization Problems under Constraints in Expectation

Abstract

SIAM Journal on Optimization, Volume 34, Issue 2, Page 1909-1936, June 2024.
Abstract. We consider sequences—indexed by time (discrete stages)—of parametric families of multistage stochastic optimization problems; thus, at each time, the optimization problems in a family are parameterized by some quantities (initial states, constraint levels, and so on). In this framework, we introduce an adapted notion of parametric time-consistent optimal solutions: They are solutions that remain optimal after truncation of the past and that are optimal for any values of the parameters. We link this time consistency notion with the concept of state variable in Markov decision processes for a class of multistage stochastic optimization problems incorporating state constraints at the final time, formulated in expectation. For such problems, when the primitive noise random process is stagewise independent and takes a finite number of values, we show that time-consistent solutions can be obtained by considering a finite-dimensional state variable. We illustrate our results on a simple dam management problem.