Nonasymptotic upper bounds on the probability of the epsilon-atypical set for Markov chains

Abstract

For a stationary, irreducible and aperiodic Markov chain with finite alphabet A, starting symbol X/sub 0/=/spl sigma/, transition probability matrix P, stationary distribution /spl pi/, support S(/spl pi/,P)={(j,k):/spl pi//sub j/P/sub k|j/>0} and for a function f such that M=/spl Delta/E/sub /spl pi/P/f(X/sub 1/,X/sub 2/) 0/{K/sub n//(1+K/sub n/)}/spl epsiv//(max/sub j,k:P(k|j)/>0|f(j,k)|]/sup 2/ where K/sub n/=(1-|A|max/sub j,k/|P/sub k|j//sup n/-/spl pi//sub k/)/n). Under the conditions stated, the set over which the sup is taken is nonempty and therefore the sup exists and is positive; it is also shown that the sup is attained at a finite value of n. A nonasymptotic version of this result is also given based on the method of Markov types.