beamer - Vrije Universiteit Amsterdam

§3.3.3 Markov Chains with Rewards
▶ Let f : I → be a reward or cost function;
▶ ∑ n
k=1 f (Xk) is the total reward up to time n;
▶ limn→∞ 1
n
∑ n
k=1 f (Xk) is the long-run average reward per unit of time;
▶ We wish to have an ergodic (or Markov-reward) property
1
lim
n→∞ n
n∑
f (Xk) = ∑
πjf (j) (w.p. 1)
k=1
j∈I
c⃝ Ad Ridder (VU) SOR– Fall 2012 28 / 36