THE CALUCULATION OF LIMIT PROBABILITIES FOR MARKOV JUMP PROCESSES

Abstract

In this paper the limit probability and the total deviation are considered by introducing an artificial transition matrix in Markov jump processes. Section 2 contains a simultaneous equation which the limit probability satisfies. In a single positive recurrent class the simultaneous equation can be reduced to an ordinary one and its solution has been given by Ballow [1], Miller [11] and Feller [5]. We note that the calculation has relation to summability methods. If the state is finite, then we can get an explicit formula of the limit probability for Markov jump processes with several classes by solving the simultaneous equation. In section 3 we shall define a total deviation from the limit probability. Our results extend that of Kemeny and Snell [9] to the denumerable state case. The notion, deviation measure, in [9] is utilized for Markov decision processes (Veinott [13]).