Constrained Markov Decision Processes with Total Expected Cost Criteria

Abstract

We study in this paper a multiobjective dynamic programmming where all the criteria are in the form of total expected sum of costs till absorption in some set of states M. We assume that instantaneous costs are strictly positive and make no assumption on the ergodic structure of the Markov Decision Process. Our main result is to extend the linear program solution approach that was previously derived for transient CMDPs (Constrained Markov Decision Processes) to general ergodic structure. Several (additive) cost metrics are defined and (possibly randomized) routing policies are sought which minimize one of the costs subject to constraints over the other objectives.