multivariate poisson hidden markov models for analysis of spatial ...

CHAPTER 2
HIDDEN MARKOV MODELS ( HMM’s) AND HIDDEN MARKOV RANDOM
FIELDS(HMRF’s)
2.1 Discrete time finite state Markov chain
Let { St
, t = 0,1,2,....} be a sequence of integer valued random variables that can assume
only an integer value {1,2,...., K }. Then { S , t = 0,1,2,....} is a K state Markov chain if
t
the probability that S
t
equals some particular value (j), given the past, depends only on
the most recent value of S . t − 1
In other words,
PS [ = j| S = iS , = m,....] = PS [ = j| S = i]
= P,
t t−1 t−2 t t−1
ij
P ij
where { }
i, j=1,2,...,
K
are the one-step transition probabilities (Srinivasan and Mehata,
1978; Ross, 1996). The transition probability, P ij
, is the probability of transitioning
from state i to state j in one time step. Note that
K
∑
j=
1
P
ij
= 1, P ≥ 0.
ij
Here, the output of the process is the set of states at each instant of time, where each
state corresponds to an observable event. The above stochastic process is called an
observable discrete time finite state Markov model.
9