Monte Carlo Methods in Statistical Mechanics: Foundations and ...

This theorem shows that the Markov chain converges as t !1to the equilibrium
distribution , irrespective ofthe initial distribution . Moreover, under the conditions
of this theorem much more can be proven | for example, a strong law of large numbers,
a central limit theorem, and a law of the iterated logarithm. For statements and proofs
of all these theorems, we refer the reader to Chung [6].
Wecannowseehowtosetupadynamic MonteCarlomethod for generating samples
from the probability distribution . It su ces to invent a transition probability matrix
P = fpxyg = fp(x ! y)g satisfying the following two conditions:
(A) Irreducibility. 3 For each pair x y 2 S, there exists ann 0 for which p (n)
xy
(B) Stationarity of . For each y 2 S,
X
x
> 0.
x pxy = y: (2.6)
Then Theorem 1 (together with its more precise counterparts) shows that simulation of
the Markovchain P constitutes a legitimateMonte Carlo method for estimatingaverages
with respect to .Wecanstart the system in any state x, andthe system is guaranteed
to converge to equilibrium ast !1[at least in the averaged sense (2.4)]. Long-time
averages of any observable f will converge with probability 1to -averages (strong
law of large numbers), and willdosowith uctuations of size n ;1=2 (central limit
theorem). In practice we will discard the data fromthe initial transient, i.e. before the
system has come close to equilibrium, but in principle this is not necessary (the bias is
of order n ;1 ,hence asymptotically much smaller than the statistical uctuations).
So far, so good! But while this is a correct Monte Carlo algorithm,itmay or may
not be an e cient one. The key di culty isthat the successivestates X0 X1 :::of the
Markov chain are correlated |perhaps very strongly | so the variance of estimates
producedfromthe dynamic Monte Carlosimulation may be much higher than in static
Monte Carlo (independent sampling). To make this precise, let f = ff(x)gx2S be a
real-valued function de ned on the state space S (i.e. a real-valued observable) that is
square-integrable with respect to . Now consider the stationary Markov chain (i.e.
start the system in the stationary distribution ,orequivalently, \equilibrate" it for a
very long time prior to observing the system). Then fftg ff(Xt)g is a stationary
stochastic process with mean
f
hfti = X
x
x f(x) (2.7)
3 Weavoid the term \ergodicity" because of its multiple and con icting meanings. In the physics
literature, and inthe mathematics literature on Markov chains with nite state space, \ergodic" is typically
used as a synonym for \irreducible" [4, Section 2.4] [5, Chapter 4]. However, in the mathematics
literature on Markov chains with general state space, \ergodic" is used as a synonym for \irreducible,
aperiodic and positive Harris recurrent" [7, p.114][8, p. 169].
5