beamer - Vrije Universiteit Amsterdam

More documents

Recommendations

Info

Unichain Case . Corollary . Suppose that the Markov chain satisfies the unichain condition with a finite set of transient states (|T| < ∞) and with an irreducible set R of aperiodic recurrent states. Then for all states i, j ∈ I . (i.e., independent of the initial state). lim n→∞ p(n) ij = πj Proof: πj = 0 for for j ∈ T; and fij = 1 for j ∈ R (see slide 7). Then apply previous slide. Note: in case of positive recurrence π is a probability distribution ( ∑ j πj = 1), see slide 24. c⃝ Ad Ridder (VU) SOR– Fall 2012 26 / 36
Summary Suppose Assumption 3.3.1 (equivalently, unichain with finite transient set T and positive recurrent set R). Suppose aperiodicity of the recurrent states. Then 1. There is a unique equilibrium distribution π = (πj)j∈I: 2. πj = 0 for j ∈ T and πj > 0 for j ∈ R; 3. π is the limiting distribution: π = πP lim n→∞ p(n) ij = πj 4. π is the long-run average distribution: 1 lim n→∞ n n∑ k=1 p (k) ij = πj c⃝ Ad Ridder (VU) SOR– Fall 2012 27 / 36
Page 1 and 2: Stochastic Operations Research Lect
Page 3 and 4: Example ⎛ ⎜ P = ⎜ ⎝ 0.6 0.4
Page 5 and 6: Recap ▶ State j transient ⇔ fjj
Page 7 and 8: Proper First-Passage Times . Coroll
Page 9 and 10: Compute Powers of P ⎛ P 129 ⎜ =
Page 11 and 12: Observations We might conclude 1. l
Page 13 and 14: Mean Return Times and Probabilities
Page 15 and 16: Probabilistic Averages II . Theorem
Page 17 and 18: Probabilistic Averages IV . Corolla
Page 19 and 20: Infinite Recurrent Sets In the situ
Page 21 and 22: Infinite Transient or Nonrecurrent
Page 23 and 24: Equilibrium Distribution . Definiti
Page 25: Limiting Probabilities . Equation (
Page 29 and 30: Finite Unichain Case . Ergodic Theo
Page 31 and 32: Expected Reward ▶ See Remark 3.3.
Page 33 and 34: Rewrite ▶ Rewrite to classic form
Page 35 and 36: Gauss-Seidel Method ▶ Construct a

beamer - Vrije Universiteit Amsterdam

Create successful ePaper yourself

Delete template?

Save as template?