Diffusion Processes with Hidden States from ... - FU Berlin, FB MI

3 TheoryThe reason why we have mentioned that it would be easy to model real life systems via Markovprocesses, is now clear: a stochastic process without memory is maybe one of the simplest onecan formulate.According to [25] and [12, 13] we will classify those processes with discrete state spaces S andparameter sets T as ”Markov chains”, and those processes with continuous S and T as ”Markovprocesses”. First Markov chains will be discussed, afterwards Markov processes. Thus we candefine Markov chains as follows:Definition 3.2 (Markov chains).A Markov chain is a set of discrete valued random variables that satisfy the followingproperty:P[X n = i n |X n−1 = i n−1 , X n−2 = i n−2 , ··· ,X 1 = i 1 , X 0 = i 0 ]= P[X n = i n |X n−1 = i n−1 ]. (3.5)For a schematic representation of simple Markov chain with three states s 1 ,s 2 ,s 3 ∈ S see Figure3.1 in the next page.p 1,1p 1,3s 1p 2,1p 1,2p 2,3p 3,2p 3,1s 2 s 3p 2,2p 3,3Figure 3.1: Scheme of a Markov chain. Schematic representation of a simple Markov chain withthree states s 1 ,s 2 ,s 3 ∈ S. The p i, j with i, j ∈ {1,2,3} are transition probabilities betweenstates i and j.14