beamer - Vrije Universiteit Amsterdam

More documents

Recommendations

Info

Finite Recurrent Sets . Corollary . Suppose that the Markov chain satisfies the unichain condition with a finite set of transient states (|T| < ∞) and with a finite irreducible set of recurrent states (|R| < ∞). Then ∑ πj = ∑ πj = 1 . Proof: for any i ∈ I and all n we have ∑ j∈I p(n) ij = 1. Thus ∑ πj = ∑ ( j∈I j∈I j∈I lim n→∞ finite set ∑ 1 = lim n→∞ n j∈I Use πj = 0 for transient j. 1 n n∑ k=1 n∑ k=1 j∈R p (k) ) ij p (k) ij 1 = lim n→∞ n n∑ ∑ Note: all states in R are positive recurrent (see also slide 20); and (πj)j∈R forms a probability mass function on R. c⃝ Ad Ridder (VU) SOR– Fall 2012 18 / 36 k=1 j∈I p (k) ij = 1
Infinite Recurrent Sets In the situation of the previous slide but with |R| = ∞. Without proof: ▶ When R is nulrecurrent: πj = 0 for all j ∈ I; ▶ When R is positive recurrent: ∑ j∈I πj = ∑ j∈R πj = 1; i.e., (πj)j∈R forms a probability mass function. c⃝ Ad Ridder (VU) SOR– Fall 2012 19 / 36
Page 1 and 2: Stochastic Operations Research Lect
Page 3 and 4: Example ⎛ ⎜ P = ⎜ ⎝ 0.6 0.4
Page 5 and 6: Recap ▶ State j transient ⇔ fjj
Page 7 and 8: Proper First-Passage Times . Coroll
Page 9 and 10: Compute Powers of P ⎛ P 129 ⎜ =
Page 11 and 12: Observations We might conclude 1. l
Page 13 and 14: Mean Return Times and Probabilities
Page 15 and 16: Probabilistic Averages II . Theorem
Page 17: Probabilistic Averages IV . Corolla
Page 21 and 22: Infinite Transient or Nonrecurrent
Page 23 and 24: Equilibrium Distribution . Definiti
Page 25 and 26: Limiting Probabilities . Equation (
Page 27 and 28: Summary Suppose Assumption 3.3.1 (e
Page 29 and 30: Finite Unichain Case . Ergodic Theo
Page 31 and 32: Expected Reward ▶ See Remark 3.3.
Page 33 and 34: Rewrite ▶ Rewrite to classic form
Page 35 and 36: Gauss-Seidel Method ▶ Construct a

beamer - Vrije Universiteit Amsterdam

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?