1 Continuous Time Processes - IPM

Lemma 1.1.2 The condition πPt = π is equivalent to πA = 0.
Proof - It is immediate that the condition πPt − π = 0 implies πA = 0.
Conversely, if πA = 0, then the Kolmogorov backward equation implies
d(πPt)
dt
= π dPt
dt = πAPt = 0.
Therefore πPt is independent of t. Substituting t = 0 we obtain πPt = πP◦ =
π as required. ♣
Example 1.1.2 We apply the above considerations to an example from
queuing theory. Assume we have a server which can service one customer
at a time. The service times for customers are independent identically distributed
exponential random variables with parameter µ. The arrival times
are also assumed to be independent identically identically distributed exponential
random variables with parameter λ. The customers waiting to be
serviced stay in a queue and we let Xt denote the number of customers in
the queue at time t. Our assumption regarding the exponential arrival times
implies
P [X(t + h) = k + 1 | X(t) = k] = λh + o(h).
Similarly the assumption about service times implies
P [X(t + h) = k − 1 | X(t) = k] = µh + o(h).
It follows that the infinitesimal generator of Xt is
⎛
−λ λ 0 0 0
⎞
· · ·
⎜
A = ⎜
⎝
µ
0
0
.
−(λ + µ)
µ
0
.
λ
−(λ + µ)
µ
.
0
λ
−(λ + µ)
.
0
0
λ
.
· · · ⎟
· · · ⎟
· · · ⎟
⎠
. ..
The system of equations πA = 0 becomes
−λπ◦ + µπ1 = 0, · · · , λπi−1 − (λ + µ)πi + µπi+1 = 0, · · ·
9