beamer - Vrije Universiteit Amsterdam

More documents

Recommendations

Info

§3.4.1 Computation of Equilibrium Probabilities ▶ Given P finite, irreducible, thus positive recurrent; ▶ Problem: compute the equilibrium distribution π; ▶ Linear system { ∑ i∈I πipij = πj (j ∈ I) ∑ j∈I πj = 1 ▶ In many applications we deal with large state spaces but sparse matrices P; Efficient to use an iterative method; ▶ Jacobi or Gauss-Seidel; c⃝ Ad Ridder (VU) SOR– Fall 2012 32 / 36
Rewrite ▶ Rewrite to classic form Ax = b; π T P = π T ⇔ (I − P T )π = 0 ▶ Delete a row (why?!) and add ∑ j∈I πj = 1. ▶ Example ⎛ 0.4 0.4 0.0 0.2 0.0 0.0 0.0 0.0 ⎞ 0.0 ⎜ 0.0 ⎜ ⎜0.5 ⎜ ⎜0.0 P = ⎜ ⎜0.0 ⎜ ⎜0.0 ⎜ ⎜0.6 ⎝0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.5 0.3 0.4 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.5 0.0 0.0 0.2 0.0 0.3 0.2 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.1 0.0 0.0 0.0 0.0 0.3 0.5 0.0 0.2 0.0 0.0 0.0 0.0 0.3 0.0 0.2 0.4 0.0 ⎟ 0.0 ⎟ 0.0 ⎟ 0.0 ⎟ 0.4 ⎟ 0.0 ⎟ 0.1⎠ 0.0 0.0 0.7 0.0 0.0 0.0 0.0 0.0 0.3 c⃝ Ad Ridder (VU) SOR– Fall 2012 33 / 36
Page 1 and 2: Stochastic Operations Research Lect
Page 3 and 4: Example ⎛ ⎜ P = ⎜ ⎝ 0.6 0.4
Page 5 and 6: Recap ▶ State j transient ⇔ fjj
Page 7 and 8: Proper First-Passage Times . Coroll
Page 9 and 10: Compute Powers of P ⎛ P 129 ⎜ =
Page 11 and 12: Observations We might conclude 1. l
Page 13 and 14: Mean Return Times and Probabilities
Page 15 and 16: Probabilistic Averages II . Theorem
Page 17 and 18: Probabilistic Averages IV . Corolla
Page 19 and 20: Infinite Recurrent Sets In the situ
Page 21 and 22: Infinite Transient or Nonrecurrent
Page 23 and 24: Equilibrium Distribution . Definiti
Page 25 and 26: Limiting Probabilities . Equation (
Page 27 and 28: Summary Suppose Assumption 3.3.1 (e
Page 29 and 30: Finite Unichain Case . Ergodic Theo
Page 31: Expected Reward ▶ See Remark 3.3.
Page 35 and 36: Gauss-Seidel Method ▶ Construct a

beamer - Vrije Universiteit Amsterdam

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?