beamer - Vrije Universiteit Amsterdam

Empirical State Averages
Recall mean return time µjj = [τjj] = [inf{n : Xn = j}|X0 = j].
.
Equation (3.3.4)
.
For recurrent states j ∈ I
. (with probability 1).
1
lim
n→∞ n
n∑
{Xk = j|X0 = j} = 1
k=1
Proof: let 0 = S0 < S1 < S2 < · · · be the consecutive returns to j. Due to the
Markov property, the interreturn periods Tn = Sn − Sn−1 are IID as τjj. Due to
recurrence, τjj is a proper random variable. Apply SLLN (or Lemma 2.2.2):
1
lim
n→∞ n
n∑
k=1
µjj
n
n
{Xk = j|X0 = j} = lim = lim ∑n n→∞ Sn n→∞
k=1 Tk
Holds also for nulrecurrent states for which µjj = ∞!
c⃝ Ad Ridder (VU) SOR– Fall 2012 12 / 36