beamer - Vrije Universiteit Amsterdam

More documents

Recommendations

Info

Finite Irreducible Sets . Corollary . .Finite irreducible sets consist of positive recurrent states only. Proof: Let C be a finite irreducible set. That is, ▶ For any i ∈ C it holds that ∑ j∈C pij = 1; thus, also for all n, ∑ j∈C p(n) ij = 1; ▶ All states in C communicate; ▶ All states have the same classification; Suppose transient or nulrecurrent, then all πj = 0. Gives a contradiction: 0 = ∑ πj = ∑ ( j∈I j∈I finite set ∑ 1 = lim n→∞ n j∈I lim n→∞ n∑ k=1 1 n n∑ k=1 p (k) ij p (k) ) ij 1 = lim n→∞ n n∑ ∑ c⃝ Ad Ridder (VU) SOR– Fall 2012 20 / 36 k=1 j∈I p (k) ij = 1
Infinite Transient or Nonrecurrent Sets ▶ Infinite irreducible sets can be transient or nulrecurrent. ▶ Example of a Random Walk. ▶ I = {0, 1, . . .}, p ∈ (0, 1), q = 1 − p. ⎛ 0 1 0 . . . ⎞ ⎜ q ⎜ P = ⎜ 0 ⎜ ⎝ 0 . 0 q 0 . p 0 q . 0 p 0 . .. . . . . . . p ⎟ . . . ⎟ ⎠ . .. ▶ If p > q the chain is transient; if q = p = 0.5 the chain is nulrecurrent; if p < q the chain is positive recurrent. c⃝ Ad Ridder (VU) SOR– Fall 2012 21 / 36
Page 1 and 2: Stochastic Operations Research Lect
Page 3 and 4: Example ⎛ ⎜ P = ⎜ ⎝ 0.6 0.4
Page 5 and 6: Recap ▶ State j transient ⇔ fjj
Page 7 and 8: Proper First-Passage Times . Coroll
Page 9 and 10: Compute Powers of P ⎛ P 129 ⎜ =
Page 11 and 12: Observations We might conclude 1. l
Page 13 and 14: Mean Return Times and Probabilities
Page 15 and 16: Probabilistic Averages II . Theorem
Page 17 and 18: Probabilistic Averages IV . Corolla
Page 19: Infinite Recurrent Sets In the situ
Page 23 and 24: Equilibrium Distribution . Definiti
Page 25 and 26: Limiting Probabilities . Equation (
Page 27 and 28: Summary Suppose Assumption 3.3.1 (e
Page 29 and 30: Finite Unichain Case . Ergodic Theo
Page 31 and 32: Expected Reward ▶ See Remark 3.3.
Page 33 and 34: Rewrite ▶ Rewrite to classic form
Page 35 and 36: Gauss-Seidel Method ▶ Construct a

beamer - Vrije Universiteit Amsterdam

Create successful ePaper yourself

Delete template?

Save as template?