10 II. The variational pr<strong>in</strong>cipleThe spectral decomposition, the Chapman-Kolmogorov equation Lemma II.1 and the assumptionP(τ)φ → φ for all φ ∈ L 2 µ −1 (Ω) show us that the i-th eigenvalue λ i (τ) is acont<strong>in</strong>uous function of τ and satisfies λ i (τ 1 +τ 2 )=λ i (τ 1 )λ i (τ 2 ) as well as λ i (0) = 1. Therefore,it can be written as λ i (τ) =e −κ iτ for some κ i > 0, thatis,ithasanexponentialdecayrate. Clearly, κ 1 =0and κ 2 ,...,κ m are close to zero. Look<strong>in</strong>g at the action of P(τ) onsome function ρ once more shows thatP(τ)ρ =∞ρ | φ i µ −1 P(τ)φ i =i=1∞e −κiτ ρ | φ i µ −1 φ i .i=1(II.27)If the time lag τ is very close to zero, almost all terms <strong>in</strong> the above sum will contribute. Ifτ becomes larger, however, there is a range of time lags τ where, for i>m,allexponentialdecay terms e −κiτ have essentially vanished, whereas all terms with i ≤ m still contribute. Ifτ <strong>in</strong>creases even further, namely if τ 1 κ 2,therearenocontributionsexcepttheveryfirstterm. For a probability distribution ρ, wehaveρ | φ 1 µ −1 = ρ | µ µ −1 =1,soP(τ)ρ ≈ µ.The system is then distributed accord<strong>in</strong>g to its equilibrium distribution and has basically“forgotten“ the <strong>in</strong>itial deviation from equilibrium expressed by ρ. Eachdom<strong>in</strong>anteigenvaluedef<strong>in</strong>es a time scale 1 κ iand a correspond<strong>in</strong>g slow process which equilibrates only if one waitsmuch longer than this time scale. The slow processes are the l<strong>in</strong>k between eigenvalues andmetastable states. A slow process usually corresponds to an equilibration process between twometastable regions, and the implied time scale 1 κ i= − τlog λ idef<strong>in</strong>es an average transition(τ)time. For this reason, the approximation of dom<strong>in</strong>ant eigenvalues is of such importance.Example 2: Consider a four state system with stochastic transition matrix⎛⎞0.7571 0.2429 0 0P= ⎜ 0.1609 0.8306 0.0085 0⎟⎝ 0 0.0084 0.8294 0.1622 ⎠ .0 0 0.2413 0.7587(II.28)Clealy, most of the transitions will occur either between states x 1 and x 2 or between x 3 andx 4 ,whereasatransitionfromx 2 to x 3 will be a very rare event. Grouped together, x 1 , x 2 andx 3 , x 4 form two metastable states. The eigenvalues are λ 1 =1, λ 2 =0.9900, λ 3 =0.5964and λ 4 =0.5894. Clearly, there is one dom<strong>in</strong>ant eigenvalue related to the slow transitionprocess between the two metastable states.Example 3: In chapter 2, we will discuss the diffusion process of a one-dimensional particle<strong>in</strong> an energy landscape V .TheparticleissubjecttoaforceF ,whichequalsthederivativeofthe energy function, but is additionally perturbed by random fluctuations. For the potentialfunction V shown <strong>in</strong> Figure II.1a, theparticlewillspendmostofthetimearoundoneofthetwo m<strong>in</strong>ima of V ,andwilljustoccasionallycrossthebarrierseparat<strong>in</strong>gthem. Figure II.1bshows a trajectory of the process, where the metastable behaviour is clearly visible. This isanother typical example of metastable states, we will <strong>in</strong>vestigate it <strong>in</strong> more detail <strong>in</strong> the next
II.2. Markov state models 11524.51.543.5130.5V(x)2.5X t02−0.51.51−10.5−1.50−1.5 −1 −0.5 0 0.5 1 1.5x−20 2 4 6 8 10Time step tx 10 4(a)(b)Figure II.1.: The potential energy function V display<strong>in</strong>g two metastable states around x = −1and x =1,andasampletrajectoryofthecorrespond<strong>in</strong>gdiffusionprocess.chapter.II.2. Markov state modelsMarkov state models (MSMs), [Pr<strong>in</strong>z et al, 2011], are a widely used technique to approximatethe eigenvalues of stochastic processes. The idea is to partition the state space <strong>in</strong>tof<strong>in</strong>itely many disjo<strong>in</strong>t sets A 1 ,...,A s with si=1 A i =Ω,andonlyconsiderthetransitionsbetween any two of these sets. This means that <strong>in</strong>stead of notic<strong>in</strong>g every transition betweenany two po<strong>in</strong>ts of the full cont<strong>in</strong>uous state space, we only take care of transitions betweenthe sets A i . Let us compute the conditional transition probability p ij (τ) between two setsA i and A j ,overtimeτ:p ij (τ) =P(X τ ∈ A j |X 0 ∈ A i )= P(X τ ∈ A j ,X 0 ∈ A i )P(X 0 ∈ A i )=A i(II.29)A jdx dy µ(x)p(x, y; τ). (II.30)A idx µ(x)The matrix P(τ) with entries p ij (τ) is a row-stochastic matrix which def<strong>in</strong>es a Markov processon the f<strong>in</strong>ite state space A 1 ,...,A s ,anditiscalledtheMSMtransfermatrix. Probabilitydensities on this space, i.e. probability vectors p =(p 1 ,...,p s ) with si=1 p i =1,aretransported by multiplication with P(τ) from the left, i.e. p τ = p 0 P(τ). As an approximationto the true eigenvalues of the cont<strong>in</strong>uous propagator, one simply computes the eigenvalues ofP(τ). Tothisend,onehastof<strong>in</strong>dthematrixentriesp ij (τ). Theseareusuallyestimatedfrom