1 Continuous Time Processes - IPM

principle construct Pt easily. The idea is to explicitly solve the Kolomogorov
(forward or backward) equation. In fact if we replace the matrices Pt and A
by scalars, we get the differential equation dp
= ap which is easily solved by
dt
p(t) = Ceat . Therefore we surmise the solution Pt = CetA for the Kolmogorov
equations where we have defined the exponential of a matrix B as the infinite
series
e B =
j
B j
j!
(1.1.5)
where B ◦ = I. Substituting tA for B and differentiating formally we see that
Ce tA satisfies the Kolmogorov equation for any matrix C. The requirement
P◦ = I (initial condition) implies that we should set C = I, so that
Pt = e tA
(1.1.6)
is the desired solution to the Kolmogorov equation. Some boundedness assumption
on A would ensure the existence of e tA , but we shall not dwell on
the issue of the existence and meaning of e B which cannot be adequately
treated in this context. An immediate implication of (1.1.6) is that
det Pt = e tTrA > 0,
assuming the determinant and trace exist. For a discrete time Markov chain
det P can be negative. It is necessary to verify that the matrices Pt fulfill the
requirements of a stochastic matrix. Proceeding formally (or by assuming
the matrices in question are finite) we show that if the matrix A fulfills the
requirements of lemma 1.1.1, then Pt is a stochastic matrix. To prove this let
A and B be matrices with row sums equal to zero, then the sum of entries
of the i th row of AB is (formally)
AijBjk =
k j
Aij Bjk = 0.
j k
From this and the definition of e B it follows that that the row sums of entries
of the matrix e tA are 1. To prove non-negativity of the entries we make use
of the formula (familiar from calculus for A a scalar)
e tA = lim
n→∞ (I + tA
n )n . (1.1.7)
4