1 Continuous Time Processes - IPM

continuous time Markov chain as the one-sided derivative
A = lim
h→0+
Ph − I
.
h
A is a real matrix independent of t. For the time being, in a rather cavalier
manner, we ignore the problem of the existence of this limit and proceed as
if the matrix A exists and has finite entries. Thus we define the derivative of
Pt at time t as
dPt
dt
= lim
h→0+
Pt+h − Pt
,
h
where the derivative is taken entry wise. The semi-group property implies
that we can factor Pt out of the right hand side of the equation. We have
two choices namely factoring Pt out on the left or on the right. Therefore we
get the equations
dPt dPt
= APt,
dt dt = PtA. (1.1.4)
These differential equations are known as the Kolmogorov backward and forward
equations respectively. They have remarkable consequences some of
which we will gradually investigate.
The (possibly infinite) matrices Pt are Markov or stochastic in the sense
that entries are non-negative and row sums are 1. Similarly the matrix A is
not arbitrary. In fact,
Lemma 1.1.1 The matrix A = (Aij) has the following properties:
Aij = 0, Aii ≤ 0, Aij ≥ 0 for i = j.
j
Proof - Follows immediately from the stochastic property of Ph and the
definition A = limh→0 Ph−I
. ♣ h
So far we have not exhibited even a single continuous time Markov chain.
Using (1.1.4) we show that it is a simple matter to construct many examples
of stochastic matrices Pt, t ≥ 0.
Example 1.1.1 Assume we are given a matrix A satisfying the properties of
lemma 1.1.1. Can we construct a continuous time Markov chain from A? If
A is an n × n matrix or it satisfies some boundedness assumption, we can in
3