From Algorithms to Z-Scores - matloff - University of California, Davis

20.1. DISCRETE-TIME MARKOV CHAINS 407
ɛ = lim
n→∞ P (En) (20.21)
= lim
n→∞ P (Bn = 1 and Dn) (20.22)
= lim
n→∞ P (Bn = 1)P (Dn) (20.23)
= 0.5(π1 + π2 + π3 + π4) (20.24)
Here we used the fact that Bn and the receiver state are independent.
Note that with the interpretation of π as the stationary distribution of the process, in Equations
(20.21) above, we do not even need to take limits.
Equations (20.21) follow a pattern we’ll use repeatedly in this chapter. In subsequent
examples we will not show the steps with the limits, but the limits are indeed there.
Make sure to mentally go through these steps yourself. 3
Now to get µ in terms of the πi note that since µ is the long-run average number of bits between
receiver replacements, it is then the reciprocal of η, the long-run fraction of bits that result in
replacements. For example, say we replace the receiver on average every 20 bits. Over a period
of 1000 bits, then (speaking on an intuitive level) that would mean about 50 replacements. Thus
approximately 0.05 (50 out of 1000) of all bits results in replacements.
µ = 1
η
(20.25)
Again suppose k = 5. A replacement will occur only from states of the form (4,j), and even then
only under the condition that the next reported bit is a 0. In other words, there are three possible
ways in which replacement can occur:
(a) We are in state (4,0). Here, since the receiver has failed, the next reported bit will definitely
be a 0, regardless of that bit’s true value. We will then have a total of k = 5 consecutive
received 0s, and therefore will replace the receiver.
(b) We are in the state (4,1), and the next bit to arrive is a true 0. It then will be reported as a
0, our fifth consecutive 0, and we will replace the receiver, as in (a).
(c) We are in the state (4,1), and the next bit to arrive is a true 1, but the receiver fails at that
time, resulting in the reported value being a 0. Again we have five consecutive reported 0s,
so we replace the receiver.
3 The other way to work this out rigorously is to assume that X0 has the distribution π, as in Section 20.1.2.4.
Then no limits are needed in (20.21. But this may be more difficult to understand.