An Improved Implementation of Discretization Algorithm for Markov Reward Models

Abstract

Discretization is one of numerical algorithms for computing transient probabilities of Markov reward models; namely continuous-time Markov chains enriched with reward structures. This algorithm is implemented in MRMC, a tool for verifying properties over probabilistic and stochastic models. MRMC uses a compressed-row representation to store sparse matrices. The representation is customized to meet MRMC's needs to store transient probability matrix and other necessary matrices. It is also used to store transient probability matrices that are constantly produced and used by discretization. However, the representation is not quite compatible for discretization when computing probabilities of high accuracy. In this paper, we propose a modification of MRMC's available compressed-row data structure to accelerate discretization computation. We also compare our method to MRMC's default by computing the transient probabilities of Markov reward models that differ by state-space size, transient matrix density, and accuracy.