A framework to design and solve Markov Decision Well-formed Net models

Abstract

The Markov decision process (MDP) (M.L. Puterman, 2005) formalism is widely used for modeling systems which exhibit both non deterministic and probabilistic behaviors (e.g. distributed systems, resource management systems, ...). Unfortunately, if the system is particularly complex then its modeling at the MDP level may be very hard; so in (M. Beccuti et al., 2007) a higher-level formalism called Markov decision well-formed net (MDWN) was proposed. The MDWN allows to describe the system in terms of its components and their interactions, while the MDP describes directly the state space and the state transitions. The MDWN model is more compact and readable: in particular, it is possible to define a complex non deterministic or probabilistic behavior as a composition of simpler non deterministic or probabilistic steps. In the MDWN formalism, the probabilistic behavior of the system is clearly distinct from the non deterministic one; actually they are designed as two separate Petri nets (PN): the probabilistic PN (Npr) and the non deterministic PN (Nnd).