Diffusion Processes with Hidden States from ... - FU Berlin, FB MI

3.2 Markov Chains, Markov Processes and Markov Propertywhere P(λ) is the a priori probability (prior), P(λ|O) the a posteriori probability (posterior) of themodel λ. P(O|λ) is the so called Likelihood L (see section 3.4 on page 21), which in fact is ameasure for the fitness of an arbitrary model to the observation.For an interesting article on Bayesian networks in computational biology see [45].3.2 Markov Chains, Markov Processes and Markov PropertyIt is unlikely that the Russian mathematician A. A. Markov during his lifetime could have an ideaof the enormous influence which his work in general, and especially his work on those stochasticprocesses that carry his name had and until today has on so numerous fields of sciences. Theconsideration on Markov processes, and everything what has in any way to do with it is somethingwhich plays an important role in modern sciences, beginning with pure and applied mathematicsand going on with theoretical physics, chemistry and biology.Those, who are interested on the life and work of A. A. Markov, are referred to [3]. In the abstractBasharin et al. for example are pointing out:”The Russian mathematician A. A. Markov (1856-1922) is known for his work innumber theory, analysis, and probability theory. He extended the weak law of largenumbers and the central limit theorem to certain sequences of dependent random variables,forming special classes of what are known as Markov chains. For illustrativepurposes Markov applied his chains to the distribution of vowels and consonants inA. S. Pushkin’s poem Eugeny Onegin.”At the beginning of our considerations on Markov processes a definition is given which on the onehand show the simplicity of the topic, and on the other hand maybe can explain, why it is so easyto model real life systems via Markov processes.Definition 3.1 (Markov Process and Markov property).Let {X(t)|t ∈ T } be a stochastic process defined on the parameter set T , with values definedon the state space S. This process is called Markov process if it satisfies the followingproperty (Markov property):P[X(t n ) = x n |X(t n−1 ) = x n−1 , X(t n−2 ) = x n−2 , ··· , X(t 0 ) = x 0 ]= P[X(t n ) = x n |X(t n−1 ) = x n−1 ], (3.4)t 0 < t 1 < t 2 < ··· < t n−2 < t n−1 < t nThe assertion of the above definition (3.1) is that the state of a particular process at any pointin time (provided that the parameter set T is denoting the time) depends only on the state ofthe process at the previous point in time. In other words: the current state includes the wholeinformation one needs, in order to determine the future distribution of the process.13