CHAPTER 2: Markov Chains (part 3)

More documents

Recommendations

Info

E i . Let z i be the proportion of women before she reach E 5 she is single given the initial state is z 0 = p 01 z 1 + p 05 0 z 1 = p 11 z 1 + p 12 z 2 + p 15 1 z 2 = p 22 z 2 + p 23 z 3 + p 24 z 4 + p 25 0 z 3 = p 32 z 2 + p 33 z 3 + p 35 0 z 4 = p 42 z 2 + p 44 z 4 + p 45 0 z 5 = 0 we have z 0 = 0.0643, z 1 = 0.0714, z 2 = 0, z 3 = 0z 4 = 0, z 5 = 0 Example [A process with short-term memory, e.g. the weather depends on the past m-days] We constrain the weather to two states s: sunny, s: cloudy - - - s c s s c s - - X n−1 X n X n+1 Suppose that given the weathers in the previous two days, we can predict the weather in the following day as sunny (yesterday) + sunny (today) =⇒ sunny (tomorrow) with probability 0.8; cloudy (tomorrow) with probability 0.2; cloudy (yesterday)+sunny (today) =⇒ sunny (tomorrow) with probability 0.6; cloudy (tomorrow) with probability 0.4; sunny (yesterday)+cloudy (today) =⇒ sunny (tomorrow) with probability 0.4; cloudy (tomorrow) with probability 0.6; cloudy (yesterday)+ cloudy (today) =⇒ sunny (tomorrow) with probability 0.1; cloudy (tomorrow) with probability 0.9; Let X n be the weather of the n’th day. Then the state space is S = {s, c} 6
We have P (x n+1 = s|x n−1 = s, x n = s) = 0.8 P (x n+1 = s|x n−1 = c, x n = s) = 0.6 Therefore {X t } is not a MC . Let Y n = (X n−1 , X n ) be the weather of the day and the previous day. Then the state space for {Y n } is S = {(s, s), (c, s), (s, c), (c, c)} Then {Y t } is a MC with transition probability matrix P = (s,s) (s,c) (c,s) (c,c) (s,s) 0.8 0.2 0 0 (s,c) 0 0 0.4 0.6 (c,s) 0.6 0.4 0 0 (c,c) 0 0 0.1 0.9 Suppose that in the past two days the weathers are (c, c). In how many days on average can we expect to have two successive sunny days? To solve the question, we define a new MC by recording the weathers in successive days. If there are two successive sunny days, we stop. Denote the process by {Z n }. The transition probability matrix is then P = (s,s) (s,c) (c,s) (c,c) (s,s) 1 0 0 0 (s,c) 0 0 0.4 0.6 (c,s) 0.6 0.4 0 0 (c,c) 0 0 0.1 0.9 Denote the states (s,s), (s, c), (c,s) and (c,c) by 1,2,3 and 4 respectively. Let v i denote the expected number of days we first have two sunny days? By the first step analysis, we have the following equations ⎧ v 1 = 0 ⎪⎨ v 2 = 1 + p 21 v 1 + p 22 v 2 + p 23 v 3 + p 24 v 4 v ⎪⎩ 3 = 1 + p 31 v 1 + p 32 v 2 + p 33 v 3 + p 34 v 4 v 4 = 1 + p 41 v 1 + p 42 v 2 + p 43 v 3 + p 44 v 4 i,e. ⎧ ⎪⎨ ⎪⎩ v 1 = 0 v 2 = 1 + 0.4v 3 + 0.6v 4 v 3 = 1 + 0.6v 1 + 0.4v 2 v 4 = 1 + 0.1v 3 + 0.9v 4 7
Page 1 and 2: CHAPTER 2: Markov Chains (part 3) E
Page 3 and 4: we have, for the others i = 1, 2, 4
Page 5: E 4 : Widowed E 5 : Died or emigrat

CHAPTER 2: Markov Chains (part 3)

Create successful ePaper yourself

Delete template?

Save as template?