A Course on Large Deviations with an Introduction to Gibbs Measures.

A <strong>Course</strong> on Large Deviations with an
Introduction to Gibbs Measures 1
Firas Rassoul-Agha
Timo Seppäläinen
Department of Mathematics, University of Utah, 155 South
1400 East, Salt Lake City, UT 84112, USA
E-mail address: firas@math.utah.edu
Mathematics Department, University of Wisconsin-Madison,
419 Van Vleck Hall, Madison, WI 53706, USA
E-mail address: seppalai@math.wisc.edu
c○Copyright 2010 Firas Rassoul-Agha and Timo Seppäläinen
1 Last updated: April 24, 2011, 20:54 MDT

2000 Mathematics Subject Classification. Primary 60F10, 82B20
Key words and phrases. large deviations, statistical mechanics, Gibbs
measures, relative entropy, variational principle, Ising model, phase
transition

To Alla, Maxim, and Kirill
To Celeste, David, Ansa, and Timo

Contents
Preface (and to the reviewers) xi
Part I. Large Deviations: general theory and i.i.d. processes
Chapter 1. Introduction 3
§1.1. Information-theoretic entropy 5
§1.2. Thermodynamic entropy 6
Chapter 2. Preliminary examples and generalities 11
§2.1. Informal large deviations 11
§2.2. Formal large deviations 12
§2.3. Lower semicontinuity and uniqueness 15
Chapter 3. More generalities and Cramér’s theorem 19
§3.1. Weak large deviation principles 19
§3.2. Cramér’s theorem 21
§3.3. Limits, deviations, and fluctuations 27
Chapter 4. Yet some more generalities 29
§4.1. Contraction principle 29
§4.2. Varadhan’s theorem and Bryc’s theorem 31
§4.3. Curie-Weiss model for ferromagnetism 37
Chapter 5. Convex analysis in large deviation theory 41
§5.1. Some elementary convex analysis 41
§5.2. Rate function as a convex conjugate 49
vii

viii Contents
§5.3. Multidimensional Cramér theorem revisited 51
Chapter 6. Relative entropy and large deviations for empirical
measures 57
§6.1. Relative entropy 58
§6.2. Sanov’s theorem 62
§6.3. Maximum entropy principle 67
Chapter 7. Large deviations for i.i.d. fields at the process level 71
§7.1. Setting 71
§7.2. Specific relative entropy 73
§7.3. Pressure and the large deviation principle 78
Part II. Statistical Mechanics
Chapter 8. Formalism for classical lattice systems 87
§8.1. Finite volume model 87
§8.2. Potentials and Hamiltonians 88
§8.3. Specifications 90
§8.4. Gibbs specifications and phase transition 93
§8.5. Observables 96
Chapter 9. Large deviations and equilibrium statistical mechanics 99
§9.1. Replacing Hamiltonians with averages 99
§9.2. Thermodynamic limit of the pressure 101
§9.3. Specific relative entropy 102
§9.4. Large deviations under Gibbs kernels 104
§9.5. Dobrushin-Lanford-Ruelle (DLR) variational principle 105
Chapter 10. Phase transition in the Ising model 111
§10.1. One-dimensional Ising model 113
§10.2. Phase transition at low temperature 115
§10.3. Uniqueness of phase at high temperature 119
§10.4. Case of no external field 121
§10.5. Case of nonzero external field 124
Chapter 11. Percolation approach to phase transition 129
Part III. Further large deviations topics
Chapter 12. Further asymptotics for i.i.d. random variables 133

Contents ix
§12.1. Refinement of Cramér’s theorem 133
§12.2. Moderate deviations 136
Chapter 13. Large deviations for Markov chains 141
§13.1. Restricting entropies on product spaces 141
§13.2. Large deviations 144
Chapter 14. Convexity criterion for large deviations 145
Chapter 15. Nonstationary independent variables 153
§15.1. Generalization of relative entropy and Sanov’s theorem 153
§15.2. Proof of the large deviation principle 155
Appendixes
Appendix A. Topics from probability 167
§A.1. Weak convergence of probability measures 167
§A.2. Ergodic theorem 170
§A.3. Stochastic ordering 175
Appendix B. Topics from analysis 177
§B.1. Measure-theoretic lemma 177
§B.2. Minimax theorem 178
Appendix C. Inequalities 183
§C.1. Holley’s inequality 183
§C.2. Griffiths’ inequality 185
§C.3. Griffiths-Hurst-Sherman inequality 186
Bibliography 189
Notation index 193
Theorems, principles, and models index 197
Author index 199
General index 201

Preface (and to the reviewers)
This book arose from courses on large deviations and related topics given by
the authors at the Departments of Mathematics at the Ohio State University
(1993), at the University of Wisconsin-Madison (2006), and at the University
of Utah (2008).
Our goal has been to create an attractive and exciting collection of material
for a semester’s course. This has two implications.
(1) First, we have not aimed at anything like an encyclopedic coverage
of different techniques for proving large deviation principles
(LDPs). Instead, our treatment centers on one classic line of reasoning:
(i) upper bound by an exponential Chebyshev inequality,
(ii) lower bound by a change of measure, and (iii) an argument to
match the rates from the first two steps. Beyond this technique
we do cover Bryc’s theorem, the subadditive method, and also an
approach based on the convexity of a local rate function due to
Baxter and Jain.
(2) Second, we have not felt obligated to stay within the boundaries
of large deviation theory but instead follow the trail of interesting
material. Large deviation theory is a natural gateway to statistical
mechanics. But we also don’t hesitate to leave large deviation theory
completely behind when we study the phase transition of the
Ising model.
Here is a brief overview of the contents of the book.
Part I covers core general large deviation theory, the relevant convex
analysis, and the large deviations of i.i.d. processes on three levels: Cramér’s
xi

xii Preface (and to the reviewers)
theorem, Sanov’s theorem, and the process level LDP for i.i.d. variables
indexed by a multidimensional square lattice.
Part II introduces Gibbs measures and proves the Dobrushin-Lanford-
Ruelle variational principle that characterizes translation-invariant Gibbs
measures in terms of the vanishing of a large deviation rate function and
in terms of a minimization problem. After this we proceed to study the
phase transition of the Ising model. We also plan to add a section on the
Fortyin-Kasteleyn random cluster model and cover the modern percolation
approach to phase transitions.
Part III develops the large deviation themes of Part I in several directions.
Large deviations of i.i.d. variables are complemented with moderate
deviations and with more precise large deviation asymptotics. From i.i.d.
processes we generalize to Markov chains and to independent but nonstationary
processes. The latter topic gives us the opportunity to introduce
an entirely different way of proving large deviation principles, namely the
Baxter-Jain theorem. This material has not previously appeared in textbooks.
We are also planning a section on large deviations for some flavor of
random walk in random environment.
The reader should note that starred exercises are used in the text later
on. Also, Dembo and Zeitouni contain thorough historical notes and references,
hence we have not attempted to present the historical record.
The ideal background for reading this book would be some familiarity
with the language of measure-theoretic probability, and with analysis
in general so that the reader is comfortable working with notions such as
lower semicontinuity or with linear spaces. In reality our courses have been
populated by students with diverse backgrounds, many with less than ideal
knowledge of analysis and probability. To make the material accessible to
these students we plan to enhance the appendixes to include an overview
of the main theorems of measure theory and probability. (And once we do
this the frequent references to basic probability textbooks that now interrupt
the flow of the prose will disappear.) The actual technical needs for
following the book are not deep and it should be possible for an instructor
to accommodate students with quick lectures on analytic technicalities
whenever needed. We have found that there is great interest in probability
theory among students of economics, engineering and sciences. This interest
should be encouraged and nurtured with accessible courses.
We are of course greatly indebted to the existing books on the subject,
especially those by Amir Dembo and Ofer Zeitouni, Jean-Dominique Deuchel
and Daniel Stroock, Frank den Hollander, and Richard Ellis. We thank Jeff
Steif for lecture notes that helped shape the proof of Theorem 10.2, Jim
Kuelbs for the material for Sections 12.1 and 12.2, and Chuck Newman for

Preface (and to the reviewers) xiii
a helpful discussion on the liquid-gas phase transition mentioned in Chapter
8. We also thank Davar Khoshnevisan for several valuable suggestions.
Support from the National Science Foundation and the Wisconsin Alumni
Research Foundation is gratefully acknowledged.
Firas Rassoul-Agha
Timo Seppäläinen
November 23rd, 2010

Part I
Large Deviations:
general theory and
i.i.d. processes

Introduction
Chapter 1
Imagine the simplest possible experiment involving randomness: tossing a
fair coin n times. When n is small, say 3 or 4, there is not much math in
there. Once the number of tosses gets large it becomes harder to analyze the
situation. However, with a large number of tosses patterns and order start
emerging from underneath the randomness: heads appear about 50% of the
time and the histogram shows a bell curve. These patterns become more
and more pronounced as the number of tosses increases. But from time to
time a random fluctuation might break the pattern: perhaps all of a sudden
10,000 tosses of a fair coin give 6000 heads. In fact, we know that there is
a chance of (1/2) 10,000 that all the tosses yield heads. The point is that to
understand the system well one cannot just be satisfied with understanding
the most likely outcomes. One also needs to understand the odds of the more
rare events. But why care about an event that has a chance of (1/2) 10,000 ?
Oversimplifying the matter for the sake of illustration, imagine that each
day at 9AM 10,000 coins are tossed simultaneously and if there are 9000 or
more heads a disaster sweeps the town. Suddenly, such a rare event is not
so unimportant. But say that each day the citizens of the town put $1 each
in the catastrophes fund. Is this enough to be able to rebuild the town when
disaster strikes? Maybe not. But if the citizens are asked to pay $1000 they
may think that is too much and simply move out of the town. Obviously,
the amount they need to pay has to do with how improbable it is to have all
coins come up heads as well as with the cost of rebuilding their town. This
is of course a caricature of how insurance premiums may be computed.
This is an introductory course on the methods of computing asymptotics
of probabilities of rare events: the theory of large deviations. Let us start
with one of the most basic computations one can actually carry out.
3

4 1. Introduction
Example 1.1. Let us consider coin tosses. Let {Xn} be an i.i.d. sequence of
Bernoulli random variables with success probability p (i.e. each Xn = 1 with
probability p and 0 otherwise). Denote the partial sum by Sn = X1+· · ·+Xn.
The law of large numbers ([15] or page 73 of [26]) says that Sn/n converges
to p, almost surely. But at any given n there is a chance of p n that we get all
heads (Sn = n) and also a chance of (1 − p) n that we get all tails (Sn = 0).
In fact, for any s ∈ (0, 1) there is always a chance that one gets a fraction
of heads close to s. Let us compute this probability.
Let us write [x] for the integral part of x ∈ R, i.e. the largest integer
smaller or equal to x. Write
P {Sn = [ns]} =
n!
[ns]!(n − [ns])! p[ns] (1 − p) n−[ns]
∼ nn p [ns] (1 − p) n−[ns]
[ns] [ns] (n − [ns]) n−[ns]
n
2π[ns](n − [ns]) ,
where we have used Stirling’s formula n! ∼ e−nnn√2πn; see, for example,
page 21 of Khoshnevisan’s textbook [26] or page 52 of Feller’s Vol. I [17].
(We say that an ∼ bn, or an is equivalent to bn, when an/bn → 1.) Let us
abbreviate
n
βn =
2π[ns](n − [ns]) ,
γn = (ns)ns (n − ns) n−ns
[ns] [ns] (n − [ns]) n−[ns]
p [ns] (1 − p) n−[ns]
pns (1 − p) n−ns .
Then, P {Sn = [ns]} is equivalent to
βnγn exp{n log n+ns log ns+n(1−s) log n(1−s)−ns log p−n(1−s) log(1−p)}.
∗ Exercise 1.2. Show that there exists a constant C such that 1
C √ n ≤ βn ≤
C and 1
Cn ≤ γn ≤ Cn for large enough n.
(1.1)
One then has
lim
n→∞
1
n log P {Sn = [ns]} = −Ip(s), with
Ip(s) = s log s
1 − s
+ (1 − s) log
p 1 − p .
This function Ip is continuous on (0, 1) and its limits at 0 and 1 are exactly
what we predicted earlier: Ip(1) = log 1
p and Ip(0) = log 1
1−p . For s ∈ [0, 1]
it is natural to set Ip(s) = ∞. Figure 1.1 shows what this function looks
like.
The function Ip in (1.1) is called a large deviation rate function. Ip(s) is
also called the entropy of the coin yielding heads with probability s relative

1.1. Information-theoretic entropy 5
∞
log 1
p
log 1
1−p
I(s)
0
0 p
1
Figure 1.1. The rate function for coin tosses.
to the one giving heads with probability p. The choice of terminology is
not a coincidence. It is indeed related to both information-theoretic and
thermodynamic entropy.
For this reason we go on a brief detour to discuss these well-known
notions of entropy and to point out the link with the large deviation rate
function Ip. The so-called relative entropy that appears in large deviation
theory will take center stage in Chapters 6–7, and again in Chapter 9 when
we discuss statistical mechanics of lattice systems.
1.1. Information-theoretic entropy
Let us regard the outcome of n coin tosses as a long word written in binary
language, a sequence of zeros and ones. The question we would like to
address is: how much information is there in a given sequence of n zeros
and ones? Or: what is the minimal amount of bits one would need to
encode such a sequence?
The answer to this question has to do with quantifying the amount of
uncertainty present in the coin itself. If the coin only gives heads, then one
knows exactly what is coming and needs 0 bits to encode the outcome. On
the other hand, if the coin is fair, then one cannot predict what will come
next in any favorable way and one thus needs exactly n bits to encode a
sequence of n zeros and ones; i.e. one needs 1 bit per character. In general,
one needs h bits per character with h ∈ [0, 1]. In fact, we have already
computed h in Example 1.1. To see how that computation is relevant we
just need to view things from a slightly different angle.
A coin that gives heads with probability s is tossed. After n tosses one
would typically get about [ns] heads and [n(1 − s)] tails. The number of
all possible combinations with [ns] heads is n
[ns] . If we are to encode this
s

6 1. Introduction
sequence with [hn] zeros and ones, then we have 2 [hn] possible words to use.
One should then have
n
≤ 2
[ns]
[hn] .
Using Stirling’s formula (or directly applying (1.1)) one obtains that the
minimal possible h is
with I 1/2 given in (1.1).
h(s) = −s log 2 s − (1 − s) log 2(1 − s) = 1 − I 1/2(s)
log 2 ,
Note that h(0) = h(1) = 0. This makes sense, since if we know s = 0
(respectively, s = 1) then we know we will only get zeros (respectively, ones).
Thus, there is nothing to encode. This is the case of complete order. On the
other hand, h(1/2) = 1. This too makes sense since there is no information
one can extract from a sequence of fair coin tosses and one needs all n
bits to encode the sequence. This is the case of complete disorder. For
s ∈ (0, 1/2), one knows that a 1 is less likely to occur than a 0 and hence
one should be able to conserve and use fewer than n bits to encode the
sequence. However, the above formula says that one cannot do better than
h(s)n. This, of course, does not tell us what the best encoding algorithm is.
1.2. Thermodynamic entropy
Once again, for the sake of illustration, we will describe an oversimplified
system. This section is inspired by Schrödinger’s course [34].
Consider a physical system of n independent identical components. By
independent we mean that the components do not communicate with each
other. By identical we mean that each of them has the same “mechanism”
attached to it, screws, pistons, and what not. Each component can be at an
energy level from the set {εℓ : ℓ ∈ N}. We submit the system to a heat bath
at a fixed absolute temperature T which causes it to have total energy E. Let
aℓ be the number of components in state εℓ. The system tries to maximize
its disorder by choosing aℓ’s so that the number of possible configurations
n!
a1!···aℓ!··· is as large as possible, subject to the constraints aℓ = n and
aℓεℓ = E.
Equivalently, one can maximize the logarithm of the quantity in question
and use Lagrange multipliers to achieve this optimization task; see page 266
of Bartle’s textbook [3]. First, one sets the gradient of
log
n!
a1! · · · aℓ! · · · − α aℓ − β aℓεℓ

1.2. Thermodynamic entropy 7
to 0. Here, α and β are the Lagrange multipliers. For convenience let us
pretend the unknowns aℓ are continuous variables. We will again use Stirling’s
formula in the form log n! ∼ n(log n − 1) then compute the derivative
of the above with respect to aℓ and get
log aℓ + α + βεℓ = 0, for all ℓ.
In other words, aℓ = Ce −βεℓ. Since one has a total number of n components,
one gets
The second constraint gives
aℓ = ne−βεℓ
−βεj e .
E = n εℓ e−βεℓ . −βεℓ e
The whole state of the system is therefore determined by the energies εℓ
and the parameter β. In principle β is a complicated function of E. However,
it turns out that β is physically the more important quantity. We will view
everything as a function of β and {εℓ} as shown in the above displays.
In thermodynamics, the entropy S of the system is defined by the relation
dQ = T dS, where dQ is the infinitesimal energy that must be given up to the
system’s surroundings as unusable heat when we do work on the pistons to
alter the system’s energy levels by dεℓ. This work is equal to aℓ dεℓ while
the corresponding increase in energy is equal to dE. dQ is the difference
between the two. But since all quantities will increase as n increases one
needs to normalize appropriately. We will hence let S be the average entropy
per component, define U = E/n as the average energy per component, let
F = log e−βεj , and write
(1.2)
dE −
aℓ dεℓ
d(βE) − E dβ − β
nT β
aℓ dεℓ
= 1
d(βU) +
T β
∂F
∂β dβ + ∂F
dεℓ
∂εℓ
dS = 1
nT
= 1
= 1
d(βU + F ).
T β
Abbreviate G = βU + F which, by the above display, has to be a function
f(S) such that f ′ (S) = T β.
Observe that G is an additive functional of the system in the following
sense. Suppose we have another system B with m independent and identical
components at energies {¯εk}. Assume the components of B are also
independent of those of A. Now consider a third system consisting of the

8 1. Introduction
two systems together and call the new system AB. This is a system of nm
components at energies {εℓ + ¯εk}. Observe that if β is kept the same for all
three systems, then FA + FB = FAB, where we have put a subscript on the
function F to indicate which system it refers to. Since U = − ∂F
∂β , the same
additive relation holds for the function G. In other words,
fA(SA) + fB(SB) = fAB(SAB).
The key observation now is that due to (1.2) entropy is also an additive
functional of the system, when T is kept fixed for all three systems. That
is, dSAB = dSA + dSB. Thus,
fA(SA) + fB(SB) = fAB(SA + SB + c).
Taking derivatives in SA and in SB one sees that f ′ A (SA) = f ′ B (SB). Since
the system B was chosen arbitrarily we see that f ′ (S) must be a universal
constant, say 1/k. This implies the familiar β = 1
kT . Moreover, G is
proportional to the entropy S.
Let us now compute G for a system that can only take one of two
energies ε1 and ε2. By symmetry, recentering, and a change of units, we can
assume that ε1 = 0 and ε2 = 1. Then, the total energy E is precisely the
number of components at energy 1. This is exactly like flipping a coin to
decide which component is at energy 1. The coin falls heads with probability
u = E/n = U = e −β /(1 + e −β ). One has
G = βu + F = u(β + F ) + (1 − u)F
= −u log u − (1 − u) log(1 − u)
= log 2 − I 1/2(u).
We see that when p = 1/2 the rate function Ip in Example 1.1 is, up to
a constant, the negative thermodynamic entropy of the two-energy system.
(This is where the customary mathematical and physical usage of the term
entropy differs: a minus sign!) This link between large deviation theory
and statistical mechanics will be explored further in the second part of this
course.
In the previous section we saw that −Ip is a linear function (with positive
slope) of h. Thus, one concludes that the thermodynamic entropy of a
physical system is essentially equal to the amount of information needed
to fully describe the system or, equivalently, to the amount of uncertainty
remaining in it.
After this introductory discussion we begin in the next chapter a systematic
study of probabilities of rare events. As in (1.1) in the coin tossing
example, these probabilities often decay exponentially in the system size.

1.2. Thermodynamic entropy 9
The identity (kβ) −1 S = U + β −1 F that represents an entropy-energy balance
will reappear several times in various guises. In can be found in Exercise
6.21, as equation (8.5) for the Curie-Weiss model, and in Section 9.5 as part
(c) of the Dobrushin-Lanford-Ruelle variational principle for lattice systems.

Preliminary examples
and generalities
2.1. Informal large deviations
Chapter 2
One common use of large deviations is to find an estimate good enough
for the purpose at hand; e.g. proving a limit theorem. The following is an
illustration of such a situation.
Example 2.1. Let {Xn} be an i.i.d. sequence with E[e θX ] < ∞ for each
θ close to 0 (i.e. |θ| < δ for some δ > 0). Assume E[X] = 0. We would
like to show that Sn/n p → 0 P -almost surely, for any p > 1/2. When
p ≥ 1 this follows from the strong law of large numbers. Let us thus assume
p ∈ (1/2, 1). Next, for t ≥ 0 Chebyshev’s inequality (see, for example, page
15 of Durrett’s textbook [15]) implies
P {Sn ≥ εn p } ≤ E[e tSn−εtnp
] = exp{−εtn p + n log E[e tX ]}.
The exponential moment assumption on X implies that E[|X| k ]t k /k! is
summable, for t ∈ [0, δ). Recalling that E[X] = 0, we see that there exists
a δ0 > 0 and a constant c such that
E[e tX ] = E[e tX − tX] ≤ 1 +
when t ∈ [0, δ0]. Then, taking t = εnp
2nc
∞
k=2
t k
k! E[|X|k ] ≤ 1 + ct 2 ,
and n large enough,
P {Sn ≥ εn p } ≤ exp{−εtn p + n log(1 + ct 2 )}
≤ exp{−εtn p + nct 2 } = exp
− ε2
4c n2p−1
.
11

12 2. Preliminary examples and generalities
Applying this to the sequence {−Xn} also gives
P {Sn ≤ −n p } ≤ exp{− ε2
4c n2p−1 }.
We have shown that P {|Sn| ≥ εn p } is summable and the Borel-Cantelli
lemma (see, e.g. page 47 of [15] or page 73 of [26]) implies that
for any ε > 0. Thus, one has
P {∃n0 : n ≥ n0 ⇒ |Sn/n p | ≤ ε} = 1,
P {∀k ∃n0 : n ≥ n0 ⇒ |Sn/n p | ≤ 1/k} = 1,
which is saying that Sn/n p → 0, P -a.s.
One can achieve the same result using martingales. In fact, one can
then weaken the moment assumption to just E[|X| 2 ] < ∞. Here is how.
Since Sn is a P -martingale (relative to the filtration σ(X1, . . . , Xn)), Doob’s
inequality (see page 250 of [15] or (8.67) of [26]) gives
P
max
k≤n |Sk| ≥ εn p
≤ 1
ε2n2p E[|Sn| 2 ] = E[|X|2 ]n
ε2n2p Pick r > 0 such that r(2p − 1) > 1. Then,
P max
k≤mr |Sk| ≥ εm pr
≤
c1
.
mr(2p−1) c
=
ε2 n−(2p−1) .
Hence, P {maxk≤m r |Sk| ≥ εm pr } is summable and the Borel-Cantelli lemma
implies that m −rp maxk≤m r |Sk| converges to 0, P -a.s.
To get the result for the full sequence observe that given n one can pick
m(n) such that (m(n) − 1) r ≤ n < m(n) r . Then,
n −p max
k≤n |Sk| ≤
m(n) r
Since m(n) r /n converges to one we are done.
2.2. Formal large deviations
n
p m(n) −rp max |Sk|.
k≤m(n) r
In the formal setting one seeks precise limits of probabilities of rare events,
typically on an exponential scale. The standardized formulation of these
limits is called a “large deviation principle” (LDP). In what follows, we
describe the setting and lead to the precise formulation.
In a very general setting one has a Hausdorff topological space X ; i.e. a
topological space where any two points can be separated by disjoint neighborhoods.
We equip X with its Borel σ-algebra B and let M1(X ) be the
space of probability measures on the resulting measurable space. We have
a sequence {µn} of such measures. In Example 1.1 on page 4 this was the
sequence of distributions µn(A) = P {Sn/n ∈ A}.

2.2. Formal large deviations 13
We are interested in the weight µn assigns to an outcome x ∈ X . In
Example 1.1 these weights decayed like e −cn for atypical points. This is the
kind of situation we want to study, and in particular we wish to compute the
constant c exactly. It could happen that c is infinite. This either happens
because x can never take place or because the probability actually decays
faster than exponentially. On the other hand, c could also be 0 which means
that x turns out to be “less rare” than we thought and the probability decays
slower than exponentially. Looking for the correct rate of decay is thus part
of the problem. Let us say then that, at scale rn ↗ ∞, the weight µn assigns
to x decays like e −crn . We still would like to compute the constant c. In
fact, this is a function of x that we will call the rate function and denote by
I(x).
This is of course not precise. Often we cannot talk about the weights
assigned to individual elements x of the space. Instead we must talk about
weights assigned to events, or subsets, of the space. But, thinking informally
for just a bit longer, on an exponential scale it makes sense to regard an event
A as rare as its least rare outcome. That is, the weight assigned to a set A
should be the maximal weight µn assigns to an element of A, i.e. e −rn infA I .
On a technical level, it is too much to expect actual convergence for all sets
A on account of boundary effects. A more reasonable formulation would be
to say that for all measurable sets A:
(2.1)
− inf I(x) ≤ lim
x∈A◦ n→∞
1
rn
log µn(A) ≤ lim
n→∞
1
rn
log µn(A) ≤ − inf I(x),
x∈A
where A ◦ and A are, respectively, the topological interior and closure of A.
Remark 2.2. The limsup for closed sets and liminf for open sets remind
us of weak convergence of probability measures where the same boundary
issue arises; see Appendix A.1 for the definition of weak convergence.
Example 2.3. Let us revisit the Bernoulli i.i.d. sequence {Xn} that we
considered in Example 1.1. If we let µn(A) = P {Sn/n ∈ A} and recall the
function Ip in (1.1), then {µn} satisfy (2.1) with normalization n and rate Ip.
Indeed, take first an open set G. For any point s ∈ G ∩ [0, 1] taking n large
enough implies that [ns]/n ∈ G and thus P {Sn/n ∈ G} ≥ P {Sn = [ns]}.
This implies that
1
lim
n→∞ n log P {Sn/n ∈ G} ≥ lim
n→∞ P {Sn = [ns]} = −Ip(s).
This inequality also holds for s ∈ G [0, 1], since Ip(s) is infinite then. Now
we can take sup over points s ∈ G and the lower bound in (2.1) follows by
taking G = A ◦ .

14 2. Preliminary examples and generalities
For a closed set F , one can split it into the union of F1 = F ∩ (−∞, p]
and F2 ∩ [p, ∞). Let us first prove the upper bound in (2.1) for F1 and F2
separately.
Let a = sup F1 ≤ p and b = inf F2 ≥ p. (If F1 is empty set a = −∞ and
if F2 is empty set b = ∞.) Assume first that a ≥ 0. Then
1
n log P {Sn/n ∈ F1} ≤ 1
n log P {Sn/n ∈ [0, a]} = 1
n log
[na]
P {Sn = k}.
k=0
∗ Exercise 2.4. Prove that P {Sn = k} increases with k ≤ [na].
By the above exercise, one sees that
1
lim
n→∞ n log P {Sn/n
1
∈ F1} ≤ lim
n→∞ n log([na] + 1)P {Sn = [na]} = −Ip(a).
This formula is still valid even when a < 0. One can prove the upper bound
in (2.1) for F2 similarly. Then, one writes
1
n log P {Sn/n ∈ F } ≤ 1
n log
P {Sn/n ∈ F1} + P {Sn/n ∈ F2}
≤ 1
1
log 2 + max
n n log P {Sn/n ∈ F1}, 1
n log P {Sn/n
∈ F2} .
Moreover, formula (1.1) or Figure (1.1) show that I is decreasing on [0, p] and
increasing on [p, 1]. Hence, infF1 Ip = Ip(a), infF2 Ip = Ip(b), and infF Ip =
min(Ip(a), Ip(b)).
Finally,
1
lim
n→∞ n log P {Sn/n ∈ F } ≤ − min(Ip(a), Ip(b)) = − inf Ip.
F
If we now take F = A, the upper bound in (2.1) follows.
We have shown that (2.1) holds with Ip defined in (1.1). This is our first
example of a full-fledged large deviation principle.
There are other instances where one can compute the rate function by
hand.
Exercise 2.5. Prove (2.1) holds for the law of the sample mean of an i.i.d.
sequence of real-valued normal random variables.
Hint: A formal computation should suggest I(x) = (x − µ) 2 /(2σ 2 ), where
µ is the mean and σ 2 is the variance.
Exercise 2.6. Prove (2.1) holds for the law of the sample mean of an i.i.d.
sequence of exponential random variables and compute the rate function
explicitly.
Hint: Use Stirling’s formula.

2.3. Lower semicontinuity and uniqueness 15
Let us continue with some general facts concerning (2.1) and rate functions.
2.3. Lower semicontinuity and uniqueness
Recall the definition of a lower semicontinuous (l.s.c.) function.
Definition 2.7. A function f : X → [−∞, ∞] is lower semicontinuous if
{f ≤ c} is closed for all c ∈ R.
Exercise 2.8. Prove that if f is lower semicontinuous then {f = −∞} is
closed.
∗ Exercise 2.9. Prove that if X is metric then f is lower semicontinuous if,
and only if, lim y→x f(y) ≥ f(x) for all x.
Say we have an arbitrary function f : X → [−∞, ∞] and would like
to produce from it a lower semicontinuous one. This is achieved by the
so-called lower semicontinuous regularization of f, which we will denote by
flsc : X → [−∞, ∞]. It is defined by
(2.2) flsc(x) = sup inf f : G ∋ x and G is open
G
.
This defines a lower semicontinuous function and in fact the maximal lower
semicontinuous minorant of f.
Lemma 2.10. flsc is lower semicontinuous and flsc(x) ≤ f(x) for all x. If
g is lower semicontinuous and satisfies g(x) ≤ f(x) for all x, then g(x) ≤
flsc(x) for all x.
Proof. flsc ≤ f is clear. To show flsc is lower semicontinuous, let x ∈
{flsc > c}. Then there is an open G containing x and such that infG f > c.
Hence by the supremum in the definition of flsc, flsc(y) ≥ infG f > c for all
y ∈ G. Thus G is an open neighborhood of x contained in {flsc > c}. So
{flsc > c} is open.
To show the last claim one just needs to show that glsc = g. For then
g(x) = sup inf g : x ∈ G and G is open
G
≤ sup inf f : x ∈ G and G is open = flsc(x).
G
We already know that glsc ≤ g. To show the other direction let c be such that
g(x) > c. Then, G = {g > c} is an open set containing x and infG g ≥ c.
Thus glsc(x) ≥ c. Now increase c to g(x).
The above can be reinterpreted in terms of epigraphs. The epigraph of
a function f is the set {(x, t) ∈ X × R : f(x) ≤ t}.

16 2. Preliminary examples and generalities
Lemma 2.11. The epigraph of flsc is the closure of that of f.
Proof. Note that the epigraph of flsc is closed. That it contains the epigraph
of f (and thus also its closure) is immediate because flsc ≤ f. For the other
inclusion we need to show that any open set outside the epigraph of f is also
outside the epigraph of flsc. Let A be such a set and let (x, t) ∈ A. There is
an open neighborhood G of x and an ε > 0 such that G × (t − ε, t + ε) ⊂ A.
So for any y ∈ G and any s ∈ (t − ε, t + ε), s < f(y). In particular,
t + ε/2 ≤ infG f ≤ flsc(x). So (x, t) is outside the epigraph of flsc.
This regularization is in fact what one gets if one only makes the necessary
changes to the values of the function to make it lower semicontinuous.
Exercise 2.12. Assume X is a metric space. Show that if xn → x, then
flsc(x) ≤ lim f(xn). Prove that for each x ∈ X there is a sequence xn → x
such that f(xn) → flsc(x). This gives the alternate definition flsc(x) =
lim y→x f(y).
Now we apply all this to rate functions of large deviation principles.
The next lemma shows that rate functions can be assumed to be lower
semicontinuous.
Lemma 2.13. Suppose I is a function such that (2.1) holds for all measurable
sets A. Then, (2.1) continues to hold if I is replaced by Ilsc.
Proof. Ilsc ≤ I and the upper bound is immediate. For the lower bound
observe that infG Ilsc = infG I when G is open.
Due to Lemma 2.13 we will call a [0, ∞]-valued function I a rate function
only when it is lower semicontinuous. Here is the precise meaning of a large
deviation principle (LDP) for the remainder of the text.
Definition 2.14. Let I : X → [0, ∞] be a lower semicontinuous function
and rn ↗ ∞ a sequence of positive constants. A sequence of probability
measures {µn} ⊂ M1(X ) is said to satisfy a large deviation principle with
rate function I and normalization rn if the following inequalities hold for all
closed F ⊂ X and all open G ⊂ X :
(2.3)
(2.4)
lim
n→∞
lim
n→∞
1
rn
1
rn
log µn(F ) ≤ − inf
F I;
log µn(G) ≥ − inf
G I.
We will abbreviate LDP(µn, rn, I) if all of the above holds. When the sets
{I ≤ c} are compact for all c ∈ R, we say I is a tight rate function.

2.3. Lower semicontinuity and uniqueness 17
Remark 2.15. In a large part of the large deviation literature a rate function
I is called good when all the sets {I ≤ c} are compact. We prefer the
term tight as more descriptive and because of the connection with exponential
tightness; see Lemma 3.3 below.
Tightness of a rate function is a very useful property. Here are two
examples.
∗ Exercise 2.16. Suppose X is a Hausdorff topological space and let E ⊂ X
be a closed set. Assume that the relative topology on E is metrized by the
metric d. Let I : E → [0, ∞] be a tight rate function and fix an arbitrary
closed set F ⊂ E. Prove that
lim
ε↘0 inf I = inf
F ε F I,
where F ε = {x ∈ E : ∃y ∈ F such that d(x, y) < ε}.
∗ Exercise 2.17. X and E as in the above exercise. Suppose ξn and ηn are
E-valued random variables defined on (Ω, F , P ), and for any δ > 0 there
exists an n0 < ∞ such that d(ξn(ω), ηn(ω)) < δ for all n ≥ n0 and ω ∈ Ω.
(a) Show that if the distributions of ξn satisfy the lower large deviation
bound (2.4) with some rate function I : E → [0, ∞], then so do the
distributions of ηn.
(b) Show that if the distributions of ξn satisfy the upper large deviation
bound (2.3) with some tight rate function I : E → [0, ∞], then so
do the distributions of ηn.
A topological space is regular if points and closed sets can be separated
by disjoint open neighborhoods. In this case, one can even be sure I is
unique.
Theorem 2.18. If X is a regular topological space, then there is at most one
(lower semicontinuous) rate function satisfying the large deviation bounds
(2.3) and (2.4).
Proof. We will show that I satisfies
I(x) = sup − lim 1
log µn(B) : x ∈ B and B is open .
One direction is easy:
− lim 1
rn
rn
log µn(B) ≤ inf
B I ≤ I(x).
For the other direction, fix x and choose c < I(x). One can separate x
from {I ≤ c} by disjoint neighborhoods. Thus, there exists an open set G

18 2. Preliminary examples and generalities
containing x and such that G ⊂ {I > c}. Then
sup − lim 1
log µn(B) : x ∈ B and B is open
rn
≥ − lim 1
log µn{G} ≥ − lim
rn
1
log µn{G} ≥ inf I ≥ c.
rn
G
Increasing c to I(x) concludes the proof.

More generalities and
Cramér’s theorem
3.1. Weak large deviation principles
Chapter 3
As we have seen in Example 1.1, when proving the lower large deviation
bound (2.4) it is enough to consider G that are local neighborhoods. It is
also enough to focus on local neighborhoods if one only needs to prove the
upper large deviation bound (2.3) for compact sets F . This often simplifies
the analysis considerably.
For the discussion in this section let X be a Hausdorff space.
Definition 3.1. A sequence of probability measures {µn} ⊂ M1(X ) is said
to satisfy a weak large deviation principle with lower semicontinuous rate
function I : X → [0, ∞] and normalization {rn} if the lower large deviation
bound (2.4) holds for all open sets G ⊂ X and the upper large deviation
bound (2.3) holds for all compact sets F ⊂ X .
With enough control on the tails of the measures µn this is in fact sufficient
for the full LDP to hold.
Definition 3.2. We say {µn} ⊂ M1(X ) is exponentially tight with normalization
rn if for each b > 0 there exists a compact set Kb such that
µn(K c b ) ≤ e−rnb for all n ∈ N.
Note that if {µn} is exponentially tight with normalization rn ↗ ∞,
then {µn} is tight; see Appendix A.1 for the definition of tightness of a
family of measures.
19

20 3. More generalities and Cramér’s theorem
Theorem 3.3. Suppose weak LDP(µn, rn, I) holds and {µn} is exponentially
tight with normalization rn. Then, LDP(µn, rn, I) holds and I is a tight rate
function.
Proof. Let F be a closed set. Write
1
1
lim log µn(F ) ≤ lim log(µn(F ∩ Kb) + µn(K
n→∞ rn
n→∞ rn
c b ))
1
≤ max − b, lim log µn(F ∩ Kb)
n→∞ rn
≤ max − b, − inf I
F ∩Kb
≤ max − b, − inf I .
F
Letting b increase to infinity proves the upper large deviation bound (2.3).
On the other hand, the weak LDP already contains the lower large deviation
bound (2.4). This, in turn, implies that
inf
Kc I ≥ − lim
b+1 n→∞
1
rn
log µn(K c b+1 ) ≥ b + 1.
Hence, {I ≤ b} is a closed subset of Kb+1 and is hence compact.
Exponential tightness helps compute one more case of large deviations
explicitly.
Exercise 3.4. Prove the large deviation principle for the law of the sample
mean Sn/n of an i.i.d. sequence of R d -valued normal random variables (with
mean µ and nonsingular covariance matrix A).
Hint: A formal computation yields I(x) = (x − µ) · A −1 (x − µ)/2. Note
that this is different from the one-dimensional case in Exercise 2.5 because
one cannot use monotonicity of I and split closed sets F into a part below
µ and a part above µ.
We end the section with an important exercise.
∗ Exercise 3.5. For x ∈ X , define upper and lower local rate functions by
(3.1)
and
(3.2)
κ(x) = − inf
G ∋ x : G open
κ(x) = − inf
G ∋ x : G open
lim
n→∞
lim
n→∞
1
rn
1
rn
log µn(G)
log µn(G).
Show that if κ = κ = κ then the weak LDP holds with rate function κ.

3.2. Cramér’s theorem 21
3.2. Cramér’s theorem
While proving the large deviation principle in several simple situations (e.g.
Example 1.1 and Exercises 2.5, 2.6, and 3.4) one notices similarities in the
arguments. Indeed, these results fall under the umbrella of a general large
deviation principle for the sample mean Sn/n of i.i.d. random variables.
(Recall that Sn = X1 + . . . + Xn.) This result, called Cramér’s theorem,
is one of the central results of large deviation theory and certainly the one
most frequently applied.
Cramér’s theorem is valid not only for real-valued random variables,
but also for R d -valued random vectors, and even for some infinite dimensional
random vectors. The infinite dimensional case will not be discussed
in the book. In this section we first develop the upper bound for the
one-dimensional case through a series of exercises. Then we state the onedimensional
Cramér’s theorem in full generality, and explore it further in
exercises. The notion of convexity makes a somewhat premature but small
appearance in this section, although it is not taken up seriously until Chapter
5.
Next we state the multidimensional theorem in full, and prove parts of
it. The final discussion of Cramér’s theorem takes place later in Section 5.3.
There, armed with some ideas from convex analysis, we prove the multidimensional
result under the natural assumption that the moment generating
function is finite in a neighborhood of the origin.
Let {Xn} be a sequence of i.i.d. real-valued random variables, and write
X for another random variable with the same distribution. Recall the moment
generating function M(θ) = E[e θX ] for θ ∈ R. Observe that M(θ) > 0
always and it can happen that M(θ) = ∞. Using Chebyshev’s inequality
(page 15 of [15]) write,
(3.3)
(3.4)
P {Sn ≥ nb} ≤ e −nθb E[e θSn ] = e −nθb M(θ) n , for θ ≥ 0,
P {Sn ≤ na} ≤ e −nθa E[e θSn ] = e −nθa M(θ) n , for θ ≤ 0.
From above we get immediately the upper bounds
1
lim
n→∞ n log P {Sn ≥ nb} ≤ − sup{θb
− log M(θ)},
θ≥0
1
lim
n→∞ n log P {Sn ≤ na} ≤ − sup{θa
− log M(θ)}.
θ≤0

22 3. More generalities and Cramér’s theorem
∗ Exercise 3.6. Suppose X has a finite mean ¯x = E[X]. Prove that if
a ≤ ¯x ≤ b, then
sup{θb
− log M(θ)} = sup{θb
− log M(θ)} and
θ≥0
θ∈R
sup{θa
− log M(θ)} = sup{θa
− log M(θ)}.
θ≤0
θ∈R
Hint: Use Jensen’s inequality (page 14 of [15] or page 40 of [26]) to show
that θb − log M(θ) ≤ 0 for θ < 0 and θa − log M(θ) ≤ 0 for θ > 0.
(3.5)
Define
I(x) = sup{θx
− log M(θ)}.
θ∈R
∗ Exercise 3.7. Prove that I is lower semicontinuous, convex, and that if
¯x = E[X] is finite then I achieves its minimum at ¯x with I(¯x) = 0.
Hint: Note that I is a supremum over lower semicontinuous and convex
functions. Show that I(x) ≥ 0 for all x, but by Jensen’s inequality (page 14
of [15] or page 40 of [26]) I(¯x) ≤ 0.
∗ Exercise 3.8. Suppose M(θ) < ∞ in some open neighborhood around
the origin. Show that then ¯x is the unique zero of I: that is, x = ¯x implies
I(x) > 0.
Hint: For any x > ¯x show that (log M(θ)) ′ < x for θ in some interval (0, δ).
Exercise 3.9. Check that the formulas for I given in Example 1.1 and
Exercises 2.5 and 2.6 match the new definition of I in (3.5).
Exercise 3.7 together with the earlier observations shows that I(x) is
nonincreasing for x < ¯x and nondecreasing for x > ¯x. In particular, if
a ≤ ¯x ≤ b, then I(a) = infx≤a I(x) and I(b) = infx≥b I(x). This proves the
upper bound for the sets F = (−∞, a] and F = [b, ∞) in the case where the
mean is finite.
Exercise 3.10. Assume d = 1. Prove that the sample mean Sn/n satisfies
the upper large deviation bound (2.3) with normalization n and rate I
defined in (3.5), with no further assumptions on the distribution.
Hint: The case of finite mean is almost done above. Then consider separately
the cases where the mean is infinite and where the mean does not
exist.
It turns out that in one dimension the full LDP with rate function I
is valid for i.i.d. random variables without any assumptions whatsoever on
the distribution. Let us state this theorem in its complete generality, even

3.2. Cramér’s theorem 23
though we will not prove it. A proof can be found in Dembo and Zeitouni
[8].
Cramér’s theorem on R. Let {Xn} be a sequence of i.i.d. real-valued
random variables. Let µn be the law of the sample mean Sn/n. Then, the
large deviation principle LDP (µn, n, I) is satisfied with I defined in (3.5).
While Cramér’s theorem is valid in general, it does not give much information
unless the variables have exponentially decaying tails. This point is
explored in the next exercise.
Exercise 3.11. Let {Xi} be an i.i.d. real-valued sequence. Assume E[X 2 1 ] <
∞ but, for all ε > 0, P {X1 > b} > e−εb for all large enough b. Show that
(a) limn→∞ 1
n log P {Sn/n > E[X1] + δ} = 0 for any δ > 0.
(b) The rate function is identically 0 on [E(X1), ∞).
Hint: For (a), deduce
P {Sn/n ≥ E[X1] + δ} ≥ P {Sn−1 ≥ (n − 1)E[X1]}P {X1 ≥ nδ + E[X1]}
and apply the central limit theorem (see page 112 of [15] or page 100 of
[26]). For (b), first find M(θ) for θ > 0. Then observe that for θ ≤ 0 and
x ≥ E[X1],
θx − log M(θ) ≤ θ(x − E[X1]) ≤ 0.
Exercise 3.12. Let {Xi} be an i.i.d. real-valued sequence. Prove that the
closure of the set {I < ∞} is the same as the closure of the convex hull of
the support of the law of X.
Hint: Let K be the latter set and y /∈ K. To show that I(y) = ∞, find
θ ∈ R such that θy − ε > sup x∈K xθ. For the other direction, take y in the
interior of {I = ∞}. To get y ∈ K, show first that φy(θ) = θy − log M(θ)
converges to infinity as θ → ∞ or −∞. Assume the former. Show that
for some ε, |x − y| ≤ ε implies φx(θ) → ∞ as θ → ∞. Then, for θ > 0,
θ(y − ε) − log M(θ) ≤ − log µ{x : |x − y| ≤ ε}. Let θ → ∞.
The information contained in Cramér’s theorem is quite crude because
only the exponentially decaying terms of a full expansion affect the result.
In some cases one can easily derive much more precise asymptotics.
Exercise 3.13. Prove that if {Xk} are i.i.d. standard normal, then for any
k ∈ N and a > 0
log P {Sn ≥ an} ∼ − a2n 1
−
2 2 log(2πna2 )
+ log 1 − 1
a2 1 × 3
+
n a4 1 × 3 × · · · × (2k − 1)
− · · · + (−1)k
n2 a2knk
.

24 3. More generalities and Cramér’s theorem
Hint: Observe that
d
e
dx
−x2 n
/2
(−1) j (1×3×· · ·×(2k−1))x
j=0
−2k−1 < −e−x2 /2 if n is even,
> −e x2 /2 if n is odd.
Exercise 3.14. Derive Cramér rates for some basic distributions.
(a) For real α > 0, the rate α exponential distribution has density f(x) =
αe −αx on R+. Derive the Cramér rate function
I(x) = αx − 1 − log αx for x > 0.
(b) For real λ > 0, the mean λ Poisson distribution has probability mass
function p(k) = e −λ λ k /k! for k ∈ Z+. Derive the Cramér rate function
I(x) = x log(x/λ) − x + λ for x ≥ 0.
We turn to discuss Cramér’s theorem in multiple dimensions. When
{Xn} are R d -valued, the moment generating function is given by M(θ) =
E[e θ·X ] for θ ∈ R d . Again, M(θ) ∈ (0, ∞]. Define
(3.6)
I(x) = sup
θ∈Rd {θ · x − log M(θ)}.
Exercise 3.15. Check that Exercises 3.7 and 3.8 apply to the multidimensional
case as well.
The full LDP of the one-dimensional Cramér theorem does not generalize
to multiple dimensions without additional assumptions on the finiteness of
the moment generating function; see [10] for counterexamples. The following
is proved in [8]; see Corollary 6.1.6 therein.
Cramér’s theorem on R d . Let {Xn} be a sequence of i.i.d. R d -valued random
variables and let µn be the law of the sample mean Sn/n. Then without
further assumptions weak LDP(µn, n, I) holds with I defined in (3.6). If,
moreover, M(θ) < ∞ in a neighborhood of 0, then LDP(µn, n, I) holds and
I is a tight rate function.
At this point, we prove the upper bound for compact sets without assumptions
on M and then the tightness of I under the assumption that M
is finite near the origin. Then we give a simple proof of the lower bound
under the restrictive assumption
(3.7)
M(θ) < ∞ for all θ ∈ R d and |θ| −1 log M(θ) → ∞ as |θ| → ∞.
Both these proofs allow us to introduce important techniques. Assumption
(3.7) ensures that the supremum in (3.6) is achieved. This is precisely the
issue that one can overcome without any assumptions when d = 1. In
Section 5.3 we revisit the theorem and prove its final version where M is
assumed to be finite in a neighborhood of the origin.

3.2. Cramér’s theorem 25
Proof of the upper bound for compacts and of the tightness of I.
The upper large deviation bound (2.3) is proved in several steps.
First, take any bounded Borel set C and write, for any θ ∈ R d ,
P {Sn/n ∈ C} = E[1I{Sn/n ∈ C}] ≤ e − infy∈C nθ·y E[e θ·Sn ]
= e −n infy∈C θ·y M(θ) n .
This shows that
1
n log P {Sn/n ∈ C} ≤ − sup inf {θ · y − log M(θ)}.
θ y∈C
We now need to interchange the sup and the inf. This can be done if C is a
compact convex set.
Minimax theorem on R d . Let C ⊂ R d be compact and convex. Let D ⊂
R d be convex. Let f : C × D → R be such that for each y ∈ C, f(y, ·) is
concave and continuous, and for each θ ∈ D, f(·, θ) is convex. Then
sup inf f(y, θ) = inf
y∈C y∈C sup f(y, θ).
θ∈D
This theorem is a special case of the more general minimax theorem
proved in Appendix B.2. To have a feeling for the above theorem think
of a horse saddle in R 3 . We have a smooth function that is convex in one
direction and concave in the other. Taking sup in the concave direction and
inf in the convex one will result in the saddle point regardless of the order.
Observe that D = {θ : M(θ) < ∞} is convex and f(y, θ) = θ·y−log M(θ)
satisfies the assumptions of the above theorem. Thus, we have that the upper
bound (even without taking any limits) is satisfied for compact convex sets.
Next, let K be any compact set and let α ≤ infK I. Fix ε > 0. Since I
is lower semicontinuous {I ≤ α − ε} is closed and one can find, for each
x ∈ K ⊂ {I > α − ε}, a closed ball Cx ⊂ {I > α − ε} and hence such
that infCx I ≥ α − ε. One can cover K with a finite collection of such balls
Cx1 , . . . , CxN . Then, applying the upper bound to these compact convex
sets one has,
N
P {Sn/n ∈ K} ≤ P {Sn/n ∈ Cxi }
i=1
θ∈D
≤ N max
1≤i≤N P {Sn/n ∈ Cxi } ≤ Ne−n(α−ε) .
Taking n → ∞ then ε → 0 then α → infK I, the upper large deviation
bound (2.3) follows for the compact set K. We have thus verified that the
upper bound in the weak LDP(µn, n, I) holds.
Next, we verify exponential tightness under the assumption that M is
finite near the origin, after which applying Theorem 3.3 would conclude the

26 3. More generalities and Cramér’s theorem
proof. To this end, from (3.3) and (3.4) it follows that for any b > 0 there
exists an a = a(b) > 0 such that
P {|S (i)
n | ≥ na} ≤ e−bn
, for i = 1, 2, . . . , d, and all n ∈ N.
d
Here for y ∈ R d , y (i) denotes its i-th coordinate. Therefore, Definition 3.2
is satisfied with rn = n and Kb = {y : |y (i) | ≤ a(b) for all i = 1, . . . , d}.
Proof of Cramér’s lower bound under (3.7). This proof introduces the
classical method of change of measure for proving LDP lower bounds.
Let G be an open set and fix x ∈ G and ε > 0 such that {y : |y − x| <
ε} ⊂ G.
Hölder’s inequality (see page 15 of [15] or page 39 of [26]) implies that
log M(θ) is a convex function: taking p = 1/t and q = 1/(1 − t)
M(tθ1 + (1 − t)θ2) = E[e tθ1·X e (1−t)θ2·X ]
≤ E[e θ1·X ] t E[e θ2·X ] 1−t = M(θ1) t M(θ2) 1−t .
Dominated convergence (see page 16 of [15] or page 46 of [26]) implies that
M(θ) is everywhere differentiable. Considering also (3.7) shows that θ · x −
log M(θ) is a concave differentiable function that achieves its maximum I(x)
at some θx. Furthermore, it must be the case that ∇M(θx) = xM(θx).
Define the probability measure νx on R d by
νx(A) =
1
M(θx) E[eθx·X 1I{X ∈ A}].
Once again, dominated convergence implies that E[e θx·X |X|] < ∞ and
E[e θx·X X] = ∇M(θx) = xM(θx);
i.e. νx has mean x. Let Qx be the law of the i.i.d. sequence {Xn} with
marginals νx. Write,
P {Sn/n ∈ G} ≥ P {|Sn − nx| < εn}
≥ e −nθx·x−nε|θx| E[e θx·Sn 1I{|Sn − nx| < εn}]
= e −nθx·x−nε|θx| M(θx) n Qx{|Sn − nx| < εn}.
The law of large numbers implies that Qx{|Sn − nx| < εn} → 1. Thus,
1
lim
n→∞ n log P {Sn/n ∈ G} ≥ −I(x) − ε|θx|.
Taking ε → 0 then sup over x ∈ G proves the lower large deviation bound
(2.4).

3.3. Limits, deviations, and fluctuations 27
3.3. Limits, deviations, and fluctuations
Let Yn be a sequence of random variables with values in a metric space
(X , d) and let µn be the law of Yn: µn(B) = P {Yn ∈ B}. Naturally an LDP
for the sequence {µn} is related to the asymptotic behavior of Yn. Suppose
LDP(µn, rn, I) holds and Yn → ¯y in probability. Then the limit ¯y does not
represent a deviation. The rate function I recognizes this with the value
I(¯y) = 0 that follows from the upper bound. For any open neighborhood G
of ¯y we have µn(G) → 1. Consequently for the closure
0 ≤ inf
¯G
I ≤ − lim r −1
n log µn( ¯ G) = 0.
Let G shrink down to ¯y. Lower semicontinuity forces I(¯y) = 0.
The reader should recognize though that the zero set of I does not
necessarily represent limit values. It may simply be that the probability of
a deviation decays slower than exponentially which again leads to I = 0.
Every rate function satisfies inf I = 0 as can be seen by taking F = X in
the upper large deviation bound (2.3).
On the other hand, we can deduce convergence of the random variables
from the LDP if the rate function has good properties. Assume that I is a
tight rate function and has a unique zero I(¯y) = 0. Let A = {y : d(y, ¯y) ≥ ε}.
Compactness and lower semicontinuity ensure that the infimum u = infA I
is achieved. Since ¯y ∈ A, it must be that u > 0. Then, for n large enough,
the upper large deviation bound (2.3) implies
P {d(Yn, ¯y) ≥ ε} ≤ e −rn(infA I−u/2) = e −rnu/2 .
Thus, Yn converges to ¯y in probability. If, moreover, rn grows fast enough,
for example like n, then the Borel-Cantelli lemma implies that Yn converges
to ¯y a.s.
For i.i.d. variables Cramér’s theorem should also be understood in relation
with the central limit theorem (CLT). Consider the case where M(θ) is
finite in a neighborhood of the origin so that there is a finite mean ¯x = E[X]
and I(x) > 0 for x = ¯x (Exercise 3.8). Then Cramér’s theorem says that
order 1 deviations of Sn/n from ¯x have exponentially vanishing probability:
for each δ > 0 there exists a constant c > 0 such that for large n
P {|Sn/n − ¯x| ≥ δ} ≤ e −cn .
By contrast, the CLT tells us that small deviations of order n −1/2 converge
to a limit distribution: for r ∈ R,
P {Sn/n − ¯x ≥ rn −1/2 } −→
n→∞
∞
r
e −s2 /2σ 2
√ 2πσ 2 ds

28 3. More generalities and Cramér’s theorem
where σ 2 is the variance of X. In the jargon of the discipline this distinction
is sometimes expressed by saying that the CLT describes fluctuations as
opposed to deviations.
There are also results about moderate deviations that fall between large
deviations and CLT fluctuations. For example, if d = 1 and M is finite in a
neighborhood of 0, then for any α ∈ (0, 1/2) one has
n −2α log P {|Sn/n − ¯x| ≥ δn −1/2+α } −→ −
n→∞ δ2
. 2
2σ
In Part III of the book in Sections 12.1 and 12.2 we discuss refinements to
Cramér’s theorem and moderate deviations.

Yet some more
generalities
4.1. Contraction principle
Chapter 4
Sometimes the problem at hand can be formulated as a mapping of another
problem on a different space. It is reasonable that an LDP for the latter
transfers to one for the former with the rate function at point y being the
smallest value of the original rate function at preimages of y. This is what
the contraction principle (or push-forward principle) is about.
Contraction principle. Suppose f is a continuous function from X to Y,
two Hausdorff topological spaces, and LDP(µn, rn, I) holds on X . Define
νn = µn ◦ f −1 ∈ M1(Y); i.e. νn(A) = µn(f −1 (A)). Define also
J(y) = inf
f(x)=y I(x),
with the convention that the inf over an empty set is infinite. Let J be the
lower semicontinuous regularization of J. That is,
J(y) = sup inf J : y ∈ G, G is open .
G
(a) LDP(νn, rn, J) holds on Y.
(b) If I is tight, then J ≡ J is tight as well.
Proof. By Lemma 2.13 it suffices to prove that J satisfies the large deviation
bounds (2.3) and (2.4). Take a closed set F ⊂ Y. Then
lim
n→∞
1
rn
log µn(f −1 (F )) ≤ − inf
x∈f −1 I(x) = − inf
(F ) y∈F inf I(x) = − inf
f(x)=y y∈F
J(y).
29

30 4. Yet some more generalities
The lower bound is proved similarly and (a) follows.
Assume now that I is tight. Observe that if J(y) < ∞, then f −1 (y) is
nonempty and closed, and the nested nonempty compact sets {I ≤ J(y) +
1/n} ∩ f −1 (y) have a nonempty intersection. Hence J(y) = I(x) for some
x ∈ f −1 (y). Consequently, { J ≤ c} = f({I ≤ c}) is a compact subset of Y.
In particular, { J ≤ c} is closed and J is lower semicontinous and is hence
identical to J.
Note that if the rate function I is not tight, then J may fail to be lower
semicontinuous (and hence J ≡ J).
Exercise 4.1. Let X = R and µn(dx) = φn(x)dx where
φn(x) = nx −2 e 1−n/x 1I (0,n)(x).
Show that {µn} are not tight on R but LDP(µn, n, I) holds with I(x) = x −1
for x > 0 and infinite otherwise; see Appendix A.1 for the definition of
tightness of measures. Note that I is not a tight rate function.
∗ Exercise 4.2. Let f : R → S 1 = {y ∈ C : |y| = 1} be f(x) = e 2πi x
x+1
and νn = µn ◦ f −1 , with µn defined in the previous exercise. Prove that
J(z) = inf f(x)=z I(x) is not lower semicontinuous and that {νn} are tight and
converge weakly to δ1. Prove also that LDP(νn, n, J) holds with J(e 2πit ) =
1−t
t for t ∈ (0, 1].
The simplest situation when the contraction principle is applied is when
X is a subspace of Y.
∗ Exercise 4.3. Suppose LDP(µn, rn, I) holds on X and that X is a Hausdorff
topological space contained in the larger Hausdorff topological space
Y. Find J so that LDP(µn, rn, J) holds on Y. What happens when I is
tight on X ?
Hint: A natural way to extend I is to simply set it to infinity outside X .
However, as demonstrated in the previous exercise, this may not be lower
semicontinuous.
Example 4.4. Let X = {0, 1}. Let {Xn} be an i.i.d. sequence of X -valued
random variables with common distribution p δ1 + (1 − p)δ0, for p ∈ (0, 1).
(This distribution is called the Bernoulli distribution and is usually denoted
by BER(p)). We have seen in Example 2.3 that if µn is the law of Sn/n =
(X1+· · ·+Xn)/n, then LDP(µn, n, Ip) holds with Ip given by (1.1). Consider
now the so-called empirical measures
Ln = 1
n
n
k=1
δXk .

4.2. Varadhan’s theorem and Bryc’s theorem 31
Ln is a random variable in Y = M1(X ). Let νn be its law. These empirical
measures usually contain more information than just the sample mean Sn/n.
In our case however
Ln = Sn
n δ1
+ 1 − Sn
δ0.
n
Hence, if one defines f : X → Y by f(s) = sδ1+(1−s)δ0, then νn = µn◦f −1 .
The contraction principle allows us to conclude a large deviation principle
for νn. The corresponding rate function is given for α ∈ M1(X ) by
H(α) = Ip(s) for α = sδ1 + (1 − s)δ0 with s ∈ [0, 1].
We will see in Chapter 6 that H(α) is a relative entropy and that the LDP
for the empirical measure holds in general for i.i.d. processes.
4.2. Varadhan’s theorem and Bryc’s theorem
A familiar fact about moment generating functions is that for a bounded
measurable function f : X → R, a probability measure µ, and a sequence
rn → ∞,
Hence
c = µ-ess sup f ≥ 1
rn log
≥ 1 log µ{f > c − ε} + c − ε.
rn
1 lim
n→∞ rn log
e rnf dµ ≥ 1
rn log
e
f>c−ε
rnf dµ
e rnf dµ = µ-ess sup f.
(µ-ess sup f = inf{b : µ(f > b) = 0}.) But what happens when µ is replaced
by a sequence µn? If {µn} satisfies a large deviation principle with
normalization rn, then the rate function I comes into the picture. The result
is known as Varadhan’s theorem. It is a probabilistic analogue of the
well-known Laplace method for asymptotics of integrals illustrated by the
next simple exercise.
Exercise 4.5. (Stirling’s formula) Use induction to show that
n! =
∞
e
0
−x x n dx.
Observe that e −x x n has a unique maximum at x = n. Prove that
lim
n→∞
n!
√ 2πn e −n n n
= 1.
Hint: Show that the main contribution to the integral comes from x ∈
[n − ε, n + ε] and use Taylor’s expansion.

32 4. Yet some more generalities
Varadhan’s theorem. Suppose LDP(µn, rn, I) holds, f : X → [−∞, ∞) is
a continuous function, and
lim
b→∞ lim 1
n→∞ rn log
e
f≥b
rnf (4.1)
dµn = −∞.
Then
1 lim
n→∞ rn log
e rnf dµn = sup(f − I).
Exercise 4.6. Show that condition (4.1) is satisfied when there exists an
α > 1 such that
sup
n
e αrnf 1/rn dµn < ∞.
This in turn is satisfied, for example, when f is bounded above.
The general idea behind the proof of the theorem is similar to the one
used when µn = µ is independent of n. Namely, if one partitions the space
into sets Ui on which µn(Ui) ∼ e−rnI(xi) for xi ∈ Ui, then
1
rn log
e rnf dµn ∼ 1
rn log e nf(xi) µn(Ui)
∼ 1
rn log e n[f(xi)−I(xi)]
∼ max[f(xi)
− I(xi)].
The actual proof follows from the next two lemmas.
Lemma 4.7. Let f : X → [−∞, ∞] be a lower semicontinuous function.
Assume {µn} satisfies the lower large deviation bound (2.4) with rate function
I and normalization rn. Then,
1 lim rn
n→∞
log
e rnf dµn ≥ sup (f − I).
f∧I −∞. Let
−∞ < c < f(x). Then G = {f > c} contains x, is open, and is such that
infG f ≥ c. Write
1 lim rn
n→∞
log
e rnf dµn ≥ lim
n→∞
1
rn log
≥ c + lim
n→∞
1
rn
G
i
e rnf dµn
log µn(G)
≥ c − inf
G I ≥ c − I(x).
Now let c grow to f(x) then take sup over x.
Lemma 4.8. Let f : X → [−∞, ∞] be an upper semicontinuous function
(i.e. −f is lower semicontinuous). Assume {µn} satisfies the upper large

4.2. Varadhan’s theorem and Bryc’s theorem 33
deviation bound (2.3) with rate function I and normalization rn. Assume
also that
lim
b→∞ lim 1
n→∞ rn log
e
f≥b
rnf dµn = −∞.
Then,
1 lim
n→∞ rn log
e rnf dµn ≤ sup (f − I).
f∧I

34 4. Yet some more generalities
Varadhan’s theorem tells us also where the largest contributions to the
integrals e rnf dµn asymptotically come from.
∗ Exercise 4.9. Suppose LDP(µn, rn, I) holds and f is a bounded continuous
function on X . Define the probability measures νn by
νn(A) = Eµn [e rnf 1IA]
E µn[e rnf ]
Prove that LDP(νn, rn, J) holds, where
J(x) = I(x) − f(x) − inf(I − f).
Hint: For F closed, replacing f by −∞ outside F gives an upper semicontinuous
function. Similarly, for G open, replacing f by −∞ outside G gives
a lower semicontinuous function.
Thus, the minimizers of the new rate function J are the largest contributers
to the integrals e rnf dµn . These are precisely the maximizers of
f − I.
As we will see in the next example, Varadhan’s theorem can provide an
explanation of the following interesting situation.
∗ Exercise 4.10. Let Zk be i.i.d. with P {Zk = 1} = P {Zk = 2} = 1/2. Set
Wn = Z1Z2 · · · Zn. Then, E[Wn] = (3/2) n , but show that Wn grows like
(3/2) n with exponentially small probability, or more precisely, that for small
enough ε > 0, P {Wn > (3/2 − ε) n } converges to 0 exponentially fast. Show
that the typical growth rate is √ 2, in the sense that P {( √ 2 − ε) n < Wn <
( √ 2 + ε) n } converges to 1 exponentially fast.
Remark 4.11. Another interesting fact about Wn is that (2/3) n Wn → 0
almost surely (by the strong law of large numbers) while (2/3) np E[W p n] → ∞
for any p > 1. This is related to an important phenomenon called intermittency.
For more about this see [? ].
Example 4.12. Let Xk = log Zk, with Zk as above. Then, Wn = e Sn . What
the above exercise shows is that even though paths {Xk} such that ( √ 2 −
ε) n < Wn < ( √ 2 + ε) n have overwhelming probability, they nevertheless do
not contribute the most to the expectation E[Wn]. Taking products seems
to have suppressed the role of these paths while amplifying the importance
of others. This is precisely what Varadhan’s theorem explains:
where
log 3
1
2 = lim
n→∞ n log E[Wn] 1 = lim
n→∞ n log E[eSn ] = sup (x − I(x)),
0≤x≤log 2
I(x) = x x log 2 − x log 2 − x
log + log + log 2
log 2 log 2 log 2 log 2
.

4.2. Varadhan’s theorem and Bryc’s theorem 35
is the Cramér rate for the law of Sn/n; see (1.1) and use the contraction
principle. The above supremum is attained at x = 2
3 log 2 instead of E[X1] =
log √ 2. Thus, the paths that contribute the most are the ones for which
Sn/n ∼ 2
3 log 2, i.e. Wn ∼ (22/3 ) n . They do occur with an exponentially
small probability, but when they do occur, they have an exponentially large
contribution. The two balance out to produce the mean value (3/2) n . LDP
and the rate function has a unique zero at x = 2
3 log 2.
The above exercises and example point to a relationship between large
deviations and statistical mechanics where one encounters integrals of the
form e nf dµn. The distributions νn in Exercise 4.9 are examples of Gibbs
measures. We will see more of this in the second part of the course. The
next Section 4.3 provides an appetizer in the context of a mean-field model.
Varadhan’s theorem gives asymptotics of integrals as a consequence of
an LDP. Perhaps not surprisingly, knowing the asymptotics of sufficiently
many integrals is equivalent to an LDP. Let Cb(X ) denote the set of bounded
and continuous functions on X .
Bryc’s theorem. Let {µn} be a sequence of probability measures on a metric
space X . Assume {µn} is exponentially tight with normalization rn.
Suppose the limit
1
Γ(f) = lim
n→∞ rn log
e rnf dµn
exists for all bounded continuous functions f. Then, LDP(µn, rn, I) holds
with the tight rate function
(4.2)
I(x) = sup
f∈Cb(X )
{f(x) − Γ(f)}.
The above theorem reminds us again of weak convergence of probability
measures.
Of course, given the LDP, Varadhan’s theorem implies that Γ(f) =
sup(f − I). Note, however, that the relation between Γ and I is not the
convex duality we will see in Chapter 5, where the functions f are continuous
linear functions. Even though until this point we have only seen convex rate
functions, the rate function in (4.2) need not be convex; recall Definition
2.14. In fact, the next section has an LDP with a nonconvex rate function.
Also, Exercise 5.27 shows how simple it is to come up with such a situation.
Proving that the limit Γ(f) exists for all bounded continuous functions
may be too hard to achieve. However, for the LDP to hold one really
needs the limit to exist for a rich enough class of functions; see for example
Theorem 4.4.10 of [8].
If X a metric vector space, then a rich enough class of functions that
would ensure the LDP via Bryc’s theorem is the class of concave Lipschitz

36 4. Yet some more generalities
functions. For interesting applications (traveling salesman, minimal spanning
trees, free energy of the short-range spin glass model, etc) where large
deviation principles are proved via this route, see [39].
Proof of Bryc’s theorem. Function I is lower semicontinuous, since it is
a sup over continuous functions. Taking f = 0 shows I(x) ≥ 0. Since {µn}
are exponentially tight, we only need to prove the weak LDP. We start with
the lower bound.
Let G be an open set and fix x ∈ G. There is a function f : X → [0, 1]
such that f(x) = 1 and f vanishes outside G. Take a > 0 and define
fa = a(f − 1). Then,
e rnfa dµn ≤ e −arn + µn(G).
Thus
max{ lim
n→∞
1
rn log µn(G), −a} ≥ lim
n→∞
Take a to infinity then sup over x.
1
rn log
e rnfa dµn = Γ(fa)
= −{fa(x) − Γ(fa)} ≥ −I(x).
For the upper bound, let C be any measurable set and let f be a bounded
continuous function. Then,
1 lim
n→∞ rn log µn(C) 1 ≤ lim
n→∞ rn log
e −rn infC
f
e rnf
dµn ≤ − inf f + Γ(f).
C
Since f is not necessarily convex a theorem like the one on page 25 cannot
be used. Let us proceed with the proof of the upper bound for an arbitrary
compact set K. Fix
c < inf I = inf
K x∈K sup {f(x) − Γ(f)}.
f∈Cb(X )
Then, for each x ∈ K there exists a bounded continuous function fx such
that fx(x) − Γ(fx) > c. Since fx is continuous, there exists an open neighborhood
of x, say Bx, such that fx(y) − Γ(fx) > c for all y ∈ Bx. One can
then cover K with a finite number of these neighborhoods, say Bx1 , . . . , BxN .
But then,
1 lim
n→∞ rn log µn(K) ≤ max
1≤k≤N lim
≤ − min
1≤k≤N
n→∞
1 log µn(Bxk )
rn
inf fxk
Bxk
− Γ(fxk ) ≤ −c.
Taking c up to infK I proves the upper bound for compact sets. The theorem
is proved.

4.3. Curie-Weiss model for ferromagnetism 37
Exercise 4.13. Let ηk > 0 be any sequence converging to 0, and let ρk be
a probability measure supported on the interval [−ηk, ηk]. Let {Xk} be a
sequence of independent random variables such that Xk has distribution ρk.
Let Sn be the partial sum as before and µn the distribution of Sn/n. Show
LDP(µn, n, I) holds with rate function I(0) = 0, I(x) = ∞ for x = 0.
We generalize this in Chapter 15. For example, the above result will continue
to hold under weak convergence ρk → δ0, as long as these distributions
have a common compact support.
But show that if Xk has distribution P {Xk = 0} = 1 − ηk and P {Xk =
k} = ηk, then the above rate function does not work if ηk converges to zero
slowly enough.
4.3. Curie-Weiss model for ferromagnetism
In this section we look at a simple model of spontaneous magnetization to
illustrate the usefulness of Cramér’s theorem and Varadhan’s theorem in
studying models from statistical mechanics.
For each n ∈ N, we have a model of n atoms, j = 1, . . . , n, each of which
has a spin ωi ∈ {−1, 1}. The space of spin configurations is denoted by
Ωn = {−1, 1} n . The energy of the system is given by the Hamiltonian
Hn(ω) = − J
2n
1≤i,j≤n
ωiωj − h
n
ωj,
where J > 0 and h ∈ R are constants. (J > 0 corresponds to ferromagnetism
and h is the strength of the external magnetic field.)
Nature prefers lower energy states, so in a ferromagnetic material the
spins tend to be aligned and follow the magnetic field, if there is any (h = 0).
The Gibbs measure for n spins is
γn(ω) = 1
Zn
j=1
e −βHn(ω) Pn(ω),
where Pn(ω) = 2 −n is the a priori measure (ω1, . . . , ωn are i.i.d. fair coin
flips), β > 0 is the inverse temperature, and Zn = e −βHn dPn is the normalization
constant called the partition function.
Remark 4.14. The more realistic Ising model in a finite box Λ ⊂ Zd has
Hamiltonian
H Ising
Λ (ω) = −J
ωxωy − h
ωx,
x,y:x−y 1 =1
where the summation is over x ∈ Λ and its nearest neighbors y. Curie-
Weiss is called the mean-field approximation of the Ising model because its
x

38 4. Yet some more generalities
Hamiltonian satisfies
Hn(ω) = − J
2
n
i=1
ωi
1
n
n
j=1
ωj
− h
where the average in parentheses is the “mean field” of the other spins. In
other words, in the Curie-Weiss model each pair of spins interacts with the
same strength regardless of the distance between them. This elimination of
spatial structure makes the analysis much easier. It is noteworthy, however,
that physical experiments show that physical structure does matter, which
makes the Ising model (Chapter 10), for example, far more accurate.
Observe that if h = 0, limβ→∞ γn(ω) = 1
2 (δω≡1 + δω≡−1). This says that
at low temperature, in the absence of external fields, energy dominates and
one gets complete order at the limit. On the other hand, limβ→0 γn(ω) = Pn,
which says that at high temperature, with no external field present, thermal
motion dominates and one gets complete disorder at the limit. This means
that, at the two extremes β = 0 and β = ∞, the finite volume system
(n < ∞) behaves differently.
Our goal is to see if this also happens in the limiting system of infinitely
many spins (n → ∞). More precisely, we want to show that there exists
a critical temperature 1/βc (called the Curie point) such that the infinite
volume model behaves differently for β < βc and β > βc.
To achieve our goal we look at the behavior of the average spin Sn/n =
1 n n i=1 ωi. In fact, this is the same as looking at the Hamiltonian:
−βHn(ω) = n Jβ
Sn
2
+ nh
2 n
Sn
n .
Let µn be the law of Sn/n under Pn (i.e. ωi’s are i.i.d.) and let νn be the
law of Sn/n under γn. The latter is the measure we are interested in.
Theorem 4.15. The following holds.
(a) If h = 0 and β ≤ 1/J, then νn converges weakly to δ0. In fact, for
all ε > 0, γn{|Sn/n| ≥ ε} → 0 exponentially fast.
(b) If h = 0 and β > 1/J, then νn converges weakly to 1
2 (δ m(β,0) +
δ −m(β,0)). In fact, if A is a closed set such that
A ∩ {m(β, 0), −m(β, 0)} = ∅,
then γn{Sn/n ∈ A} → 0 exponentially fast. Here, z = ±m(β, 0)
= Jβz.
are the two nonzero solutions of 1
2
log 1+z
1−z
(c) If h = 0, then νn converges weakly to δm(β,h). In fact, for all ε > 0,
γn{|Sn/n − m(β, h)| ≥ ε} → 0 exponentially fast. Here, m(β, h) is
the unique solution of 1 1+z
2 log 1−z = Jβz + βh that has the same sign
as h.
n
i=1
ωi

4.3. Curie-Weiss model for ferromagnetism 39
h
0
δm, m > 0
1
2 ( δ− m+ δm ) , m > 0
T c = J
δm, m < 0
Figure 4.1. The Curie-Weiss phase diagram. T = 1/β and m =
m(β, h) is as in Theorem 4.15.
When there is no external field, the theorem says that if the temperature
is above J then neither −1 nor +1 gains the upper hand and “randomness
beats couplings”. In other words, when we heat up a magnet, it loses its
magnetization. When, on the other hand, the temperature is below J, “coupling
tendency wins over noise” and there are two possible limits for the
sample mean, with equal probability. In other words, when cooled down a
ferromagnetic material gets magnetized even in the absence of an external
magnetic field. Of course, the ferromagnetic material also gets magnetized if
we switch the external magnetic field on. The theorem then says that if the
temperature is low and we switch the magnetic field off, the magnet does
not lose its magnetization. This is known as spontaneous magnetization.
See Figure 4.1 for the phase diagram summarizing these results. For more
on this, see the discussion on page 93.
Proof of Theorem 4.15. By Cramér’s theorem on R (page 23) we know
that LDP(µn, n, I) holds with rate
I(z) =
1−z
2
log(1 − z) + 1+z
2
δ0
T
log(1 + z) if − 1 ≤ z ≤ 1,
∞ otherwise.
Exercise 4.16. Check the above formula for the rate function by direct
computation of (3.6) and also by an application of the contraction principle
to the Bernoulli case in Example 2.3.

40 4. Yet some more generalities
Next write
νn(B) = γn{Sn/n ∈ B} = 1
= 1
Zn
= 1
1IB(z)e
Zn
n
Jβ
1IB(Sn/n)e −βHn dPn
Zn
1IB(Sn/n)e n
Jβ
2 (Sn/n)2 +βh(Sn/n)
2 z2 +βhz
By Exercise 4.9, LDP(νn, n, I) holds with rate
µn(dz).
z
dPn
I(z) = I(z) − Jβ
2 z2 Jβ
− βhz − c, where c = inf{I(z)
− 2 z2 − βhz}.
Exercise 4.17. Prove the following.
(a) If h = 0, then I is uniquely minimized at z = m(β, h), the unique
= Jβz + βh that is of the same sign as h.
solution of 1
2
log 1+z
1−z
(b) If h = 0 and Jβ ≤ 1, then I is convex and is uniquely minimized
at z = 0. In other words, m(β, 0) = 0.
(c) If h = 0 and Jβ > 1, then I is not convex and is minimized at
= Jβz.
{−m(β), +m(β)}, the two nonzero solutions of 1
2
log 1+z
1−z
Now, if h = 0 and βJ ≤ 1, then for any ε > 0, γn{|Sn/n| ≥ ε} → 0
exponentially fast because by the upper large deviation bound (2.3),
1
lim
n→∞ n log γn{|Sn/n| ≥ ε} ≤ − inf I(z) < 0.
|z|≥ε
This implies that νn converges weakly to δ0 and concludes the proof of (a).
(c) is proved similarly.
If h = 0, βJ > 1, and A is closed such that A ∩ {m(β), −m(β)} = ∅,
then
1
lim
n→∞ n log γn{Sn/n ∈ A} ≤ − inf I < 0
A
and γn{Sn/n ∈ A} → 0 exponentially fast. In particular,
lim
n→∞ γn{|Sn/n − m(β)| < ε or |Sn/n + m(β)| < ε} = 1.
But (ωi) n i=1 and (−ωi) n i=1 have the same distribution. Hence, νn is symmetric
and
γn{|Sn/n − m(β)| < ε} = γn{|Sn/n + m(β)| < ε}.
Thus, both limits exist and are equal to 1/2. This shows that νn converges
weakly to 1
2 (δ m(β) + δ −m(β)) and concludes the proof of (b).

Convex analysis in
large deviation theory
Chapter 5
We have seen in the previous chapters how convex sets and convex functions
occur naturally in the study of large deviations. This chapter will, therefore,
be devoted to the analysis of such objects. Although the results in this
chapter may seem remote, they are an essential tool in large deviation theory.
5.1. Some elementary convex analysis
Recall how Lemmas 2.13 and 2.10 showed how to replace a function by the
“best possible” lower semicontinuous version. Say now we would like to also
require convexity. The natural way to do this would be by taking sup over
affine minorants. The epigraph of the resulting function will then be the
closure of the convex hull of the epigraph the original function. Moreover,
if f were convex to begin with, this construction should again give the
lower semicontinuous regularization, defined in (2.2) as the maximal lower
semicontinuous minorant. Let us make all this precise.
The setting is that of two real vector spaces in duality. That is, we
have vector spaces X and Y and a bilinear map 〈·, ·〉 : X × Y → R. The
weak topology σ(X , Y) on X is the minimal topology making the maps
{x ↦→ 〈x, y〉 : y ∈ Y} continuous. Similarly for σ(Y, X ) on Y.
Assumption 5.1. We will assume that for each nonzero x ∈ X there exists
a y ∈ Y such that 〈x, y〉 = 0. We also assume that for each nonzero y ∈ Y
there exists an x ∈ X such that 〈x, y〉 = 0.
41

42 5. Convex analysis in large deviation theory
With the above assumption in force the topologies σ(X , Y) and σ(Y, X )
make X and Y Hausdorff topological spaces. We will have to check this
assumption every time we put two spaces in duality.
Example 5.2. If X = Y = R d , then σ(X , Y) = σ(Y, X ) is the Euclidean
topology on R d .
Example 5.3. If X is any Banach space and Y = X ∗ , then σ(X , Y) and
σ(Y, X ) are, respectively, the weak and weak ∗ topologies from functional
analysis.
Example 5.4. Our most important applications will be when X = M(S)
is the space of real-valued Borel measures on some topological space S and
Y is some vector space of bounded Borel functions on S, e.g. Cb(S) when S
is metric.
In the following proposition we give a few consequences. We leave the
proof of this proposition to the reader, either as an exercise or to be looked
up from a functional analysis text. For example, Chapter 3 in Rudin’s
textbook [33] has a section on weak topologies.
Proposition 5.5. Setting as above.
(a) A base for σ(X , Y) is given by the collection of sets of the type
{x ∈ X : |〈x, yi〉 − 〈x0, yi〉| < ε, i = 1, . . . , m}
for m ∈ N, x0 ∈ X , y1, . . . , ym ∈ Y, and ε > 0. In particular,
σ(X , Y) is locally convex and thus X is locally convex, which means
that every neighborhood of x ∈ X contains a convex neighborhood
of x.
(b) xn → x in σ(X , Y) if and only if 〈xn, y〉 → 〈x, y〉 for all y ∈ Y.
(c) The maps R × X → X : (a, x) ↦→ ax and X × X → X : (x, x ′ ) ↦→
x + x ′ are continuous in the appropriate topologies. This says that
X is a topological vector space.
(d) Suppose f : X → R is a continuous linear functional. Then, there
exists a unique y ∈ Y such that f(x) = 〈x, y〉 for all x ∈ X .
This gives an isomorphism between the dual X ∗ (the vector space
of continuous linear functionals on X ) and Y.
(e) All of the above also holds if the roles of X and Y are reversed.
Here is a fundamental fact that we will need in what follows.
Hahn-Banach separation theorem. Suppose A and B are nonempty,
disjoint, convex subsets of a locally convex topological real vector space Z.

5.1. Some elementary convex analysis 43
(a) If A is open there exists z ∗ ∈ Z ∗ and γ ∈ R such that
〈z ′ , z ∗ 〉 < γ ≤ 〈z ′′ , z∗〉, ∀z ′ ∈ A, z ′′ ∈ B.
(b) If A is compact and B is closed then there exists z ∗ ∈ Z ∗ and
γ1, γ2 ∈ R such that
〈z ′ , z ∗ 〉 < γ1 < γ2 < 〈z ′′ , z∗〉, ∀z ′ ∈ A, z ′′ ∈ B.
For the proof see Theorem 3.4 of [33].
Definition 5.6. For any function f : X → [−∞, ∞], define the convex
conjugate f ∗ : Y → [−∞, ∞] by
f ∗ (y) = sup{〈x,
y〉 − f(x)}.
x∈X
Similarly, if g : Y → [−∞, ∞], define g ∗ : X → [−∞, ∞] by
g ∗ (y) = sup{〈x,
y〉 − g(y)}.
y∈Y
Inductively, f ∗∗ = (f ∗ ) ∗ is the convex biconjugate and f ∗n = (f ∗(n−1) ) ∗ .
Similarly for g.
Proposition 5.12 below shows why we use the word “convex” in the above
terminology.
f ∗∗ is in fact the object we are after, as explaind in the first paragraph
of this section.
∗ Exercise 5.7. We say f is affine if f(x) = a + 〈x, y〉 for some a ∈ R and
y ∈ Y. Prove that f ∗∗ is the sup over the affine minorants of f.
Next, recall the definition of a convex function.
Definition 5.8. We say a function f : X → [−∞, ∞] is convex if
f(αx1 + (1 − α)x2) ≤ αf(x1) + (1 − α)f(x2)
for all x1, x2 ∈ X and α ∈ [0, 1] such that the right-hand-side is well defined.
A convex function is proper if it maps into (−∞, ∞] and is not identically
∞.
∗ Exercise 5.9. Suppose E is a convex subset of X and let f : E → (−∞, ∞]
be lower semicontinuous. Prove that f is convex on E if and only if for all
x, y ∈ E,
x + y
f ≤
2
f(x) + f(y)
.
2
Hint: First take care of dyadic rationals α = k2 −n by induction on n.

44 5. Convex analysis in large deviation theory
Exercise 5.10. Show that a finite convex function on an open interval is
continuous. This is also true more generally in finite-dimensional spaces.
See Theorem 10.1 in [32].
∗ Exercise 5.11. (Fenchel-Young inequality) Prove that for any x ∈ X and
y ∈ Y
〈x, y〉 ≤ f(x) + f ∗ (y),
whenever the right-hand-side makes sense.
Proposition 5.12. For f arbitrary the following is true.
(a) f ∗ is convex and lower semicontinuous, where the constants f ∗ ≡ ∞
and f ∗ ≡ −∞ also qualify as convex lower semicontinuous.
(b) f ∗∗ (x) ≤ f(x) for all x ∈ X .
(c) f ∗n = f ∗ if n ≥ 1 is odd and f ∗n = f ∗∗ if n ≥ 2 is even.
Proof. If f takes the value −∞, then f ∗ ≡ ∞. If f ≡ ∞, then f ∗ ≡ −∞.
Thus, part (a) is true if f is not proper. If, on the other hand, f is proper,
then f ∗ is the pointwise sup over a nonempty family of affine continuous
functions y ↦→ 〈x, y〉 − f(x), x ∈ X with f(x) < ∞. Hence the claim of part
(a) is true in this case too. Part (b) follows from Exercise 5.7.
Observe next that if f1 ≤ f2, then f ∗ 1 ≥ f ∗ 2 . Thus, f ∗∗ ≤ f implies that
f ∗3 ≥ f ∗ . But part (b) implies f ∗3 ≤ f ∗ . Hence, f ∗3 = f ∗ and part (c)
follows by induction.
If f ∗∗ is supposed to be a convex lower semicontinuous regularization of
f, it should be the case that if f is already convex and lower semicontinuous,
then f ∗∗ simply recovers f.
Fenchel-Moreau theorem. Consider a function f : X → (−∞, ∞] that
is not identically ∞. Then, f = f ∗∗ if and only if f is convex and lower
semicontinuous.
Proof. If the equality holds, the previous proposition implies that f is convex
and lower semicontinuous. Let us now prove the other direction. Again,
the previous proposition implies that f ∗∗ ≤ f. So we need to show that
f ∗∗ ≥ f for a convex lower semicontinuous function f. Let us start by
showing that f ∗ is proper.
To this end, pick t0 > −∞ and x0 ∈ X such that t0 < f(x0) < ∞. We
then have that
f ∗ (y) ≥ 〈x0, y〉 − f(x0) > −∞
for all y ∈ Y. Thus, we only need to show the existence of y0 ∈ Y such
that f ∗ (y0) < ∞. Since f is convex lower semicontinuous, its epigraph
{(x, t) ∈ X × R : f(x) ≤ t < ∞} is a closed and convex set. Since X × R is a

5.1. Some elementary convex analysis 45
locally convex topological vector space, and (x0, t0) is outside the epigraph,
the Hahn-Banach separation theorem implies that there exists a φ ∈ (X ×R) ∗
and γ ∈ R such that
φ(x0, t0) > γ > φ(x, t)
for all (x, t) in the epigraph.
Exercise 5.13. Prove that (X × R) ∗ is isomorphic to X ∗ × R (and hence
to Y × R).
By the above exercise, there exists a pair (y, s) ∈ Y × R such that
φ(x, t) = 〈x, y〉 + st and
〈x0, y〉 + st0 > γ > 〈x, y〉 + st
for all (x, t) in the epigraph. In particular, since (x0, f(x0)) is in the epigraph,
one has
〈x0, y〉 + st0 > 〈x0, y〉 + sf(x0)
and thus st0 > sf(x0) and s < 0. But then one has 〈x, y/|s|〉 − f(x) < γ/|s|
whenever f(x) < ∞. It follows that
f ∗ (y/|s|) = sup {〈x, y/|s|〉 − f(x)} ≤ γ/|s| < ∞.
x:f(x) −∞
and f ∗∗ ≤ f is not identically ∞. We are now ready to prove that f ∗∗ ≥ f.
Suppose there exists a point x0 ∈ X such that f ∗∗ (x0) < f(x0). Then
|f ∗∗ (x0)| < ∞ and we can separate (x0, f ∗∗ (x0)) from the epigraph of f. In
other words, there exists a γ ∈ R and a pair (y, s) ∈ Y × R such that
〈x0, y〉 + sf ∗∗ (x0) > γ > 〈x, y〉 + st
for all (x, t) in the epigraph of f. Now pick x such that f(x) < ∞ and let t
grow to infinity. This proves that s ≤ 0. Suppose s < 0. Then,
f ∗∗ (x0) + f ∗ (−y/s) = f ∗∗ (x0) + sup {〈x, y/|s|〉 − f(x)}
x:f(x) 〈x, y〉

46 5. Convex analysis in large deviation theory
for all x such that f(x) < ∞. Observe that this implies that f(x0) cannot
be finite. Now let α > 0. Since f ∗ is proper we can choose y0 such that
f ∗ (y0) < ∞. One has
But then
f ∗ (y0 + αy) = sup{〈x,
y0 + αy〉 − f(x)}
x
≤ sup{〈x,
y0〉 − f(x)} + α sup
x
≤ f ∗ (y0) + αγ.
x:f(x) γ/s − 〈x, y/s〉

5.1. Some elementary convex analysis 47
for all x such that f(x) < ∞ and thus, in fact, for all x. Since the righthand-side
is convex and continuous in x, this implies that
t0 ≥ f ∗∗ (x0) ≥ γ/s − 〈x0, y/s〉
which contradicts the separation. So s = 0 and
〈x, y〉 < γ < 〈x0, y〉
for all x such that f(x) < ∞. Thus f(x) is infinite for all x with 〈x, y〉 > γ.
But resetting f ∗∗ to infinity on this region creates a convex lower semicontinuous
minorant of f. Thus f ∗∗ is also infinite on this region and in particular
at x0. This contradicts the choice of x0.
If f is already convex, then f ∗∗ is nothing but the lower semicontinuous
regularization of f, which according to Lemma 2.10 is the maximal lower
semicontinuous minorant of f.
Theorem 5.18. Consider f : X → (−∞, ∞] that is not identically infinite.
Recall the lower semicontinuous regularization
flsc(x) = sup inf f : x ∈ G and G is open
G
.
If f is convex, then flsc = f ∗∗ .
Proof. Note that if f ∗∗ takes the value −∞, then f ∗∗∗ = ∞ and thus
f ∗ = ∞ and f ∗∗ = −∞. By Lemma 5.17, this implies that the closure of
the epigraph of f is all of X ×R and thus, by Lemma 2.11, that the epigraph
of flsc is all of X × R and flsc = −∞ = f ∗∗ . We can, therefore, assume that
f ∗∗ > −∞.
Since f ∗∗ ≤ f and f ∗∗ is lower semicontinuous Lemma 2.10 implies then
that f ∗∗ ≤ flsc. In particular, flsc > −∞.
If we now show that flsc is convex, then Corollary 5.15 implies that
flsc ≤ f ∗∗ . Let G be an open neighborhood of αx1 + (1 − α)x2 such that
infG f > −∞. By the continuity of the map (x1, x2) ↦→ αx1 + (1 − α)x2
there exists a neighborhood G1 of x1 and a neighborhood G2 of x2 such that
αG1 + (1 − α)G2 ⊂ G. Then,
inf f(αz1 + (1 − α)z2) ≤ α inf f + (1 − α) inf f
z1∈G1 z2∈G2
G1
G2
≤ αflsc(x1) + (1 − α)flsc(x2).
inf
G f ≤ inf
Taking supremum over G proves the convexity of flsc and concludes the
proof of the theorem.
Let us now go back to the picture of f ∗∗ as the supremum of affine
minorants of f. To be able to visualize things, let us look at the case X = R d .
Let f : R d → R be a convex function and suppose f ∗ (y) = y · x − f(x) for

48 5. Convex analysis in large deviation theory
some x and y in R d . Then, for all u, y·u−f(u) ≤ y·x−f(x) or, equivalently,
f(u) ≥ f(x) + y · (u − x). This says that the graph {(u, f(u)) : u ∈ R d } ⊂
R d+1 has a tangent hyperplane at (x, f(x)) with normal (y, −1). If f is
differentiable, then y is unique and equal to ∇f(x). Otherwise there may
be several tangent hyperplanes at (x, f(x)). This discussion remains valid
in the more general setting of this section.
Definition 5.19. Let f : X → [−∞, ∞]. The subdifferential of f at x is
the multivalued mapping ∂f : X → Y defined by
∂f(x) = {y ∈ Y : ∀u ∈ X , f(u) ≥ f(x) + 〈u − x, y〉}.
Exercise 5.20. Prove that if f is not identically ∞ and f(x) = ∞, then
∂f(x) = ∅. Then prove that if f is convex and proper and if x is in the
interior of {u : f(u) < ∞}, then ∂f(x) = ∅. Find a counterexample where
f(x) < ∞ but ∂f(x) = ∅.
Hint: Use the Hahn-Banach separation theorem.
Exercise 5.21. Let X = R d and assume f is convex, proper, and differentiable
at x. Prove that ∂f(x) = {∇f(x)}.
Theorem 5.22. Let f : X → (−∞, ∞] be not identically infinite. Then the
following are equivalent.
(a) y ∈ ∂f(x);
(b) f(x) + f ∗ (y) ≤ 〈x, y〉;
(c) f(x) + f ∗ (y) = 〈x, y〉.
If f is also convex and lower semicontinuous, then (a)-(c) are equivalent to
(d) x ∈ ∂f ∗ (y);
Proof. (b) implies (c) by the Fenchel-Young inequality (Exercise 5.11). (c)
implies (a) implies (b) by the definition of f ∗ . Applying the equivalence of
(a) and (c) to f ∗ one sees that (d) is equivalent to f ∗ (y) + f ∗∗ (x) = 〈x, y〉.
But f = f ∗∗ if f is convex, proper, and lower semicontinuous. Hence (d) is
equivalent to (c).
Exercise 5.23. Let f be a finite convex function on (a, b). The following
properties follow quickly from the definition.
(a) ∂f(x) is a nonempty closed interval. f ′ (x) exists if and only if
∂f(x) is a singleton, and then ∂f(x) = {f ′ (x)}.
(b) Let x = y and α ∈ ∂f(x). Then f(y) = f(x) + α(y − x) implies
that α ∈ ∂f(y).

5.2. Rate function as a convex conjugate 49
(c) If x < y, α ∈ ∂f(x), and β ∈ ∂f(y) then
α ≤
f(y) − f(x)
y − x
≤ β.
∂f(x) and ∂f(y) have at most one point in common. If that happens
then this point is an endpoint for both sets and over [x, y] the
graph of f is a line segment.
(d) Suppose xj > x, αj ∈ ∂f(xj) and xj → x. Then αj → sup ∂f(x).
5.2. Rate function as a convex conjugate
We will now prove a general theorem about the upper large deviation bound
(2.3) for compact sets.
Let X and Y be two vector spaces in duality and let X be topologized
by σ(X , Y). Let E be a closed convex subset of X . We are given a sequence
of probability measures {µn} ⊂ M1(E). Define
1
¯p(y) = lim
n→∞ rn log
e rn〈x,y〉 µn(dx) ∈ [−∞, ∞] for y ∈ Y
and
E
¯p ∗ (x) = sup{〈x,
y〉 − ¯p(y)} ∈ [0, ∞] for x ∈ X .
y∈Y
Theorem 5.24. ¯p is convex and ¯p ∗ : X → [0, ∞] is convex and lower
semicontinuous. Moreover, for any closed compact subset F ⊂ E,
lim
n→∞
1
rn log µn(F ) ≤ − inf
F ¯p ∗ .
Proof. ¯p is convex by Hölder’s inequality (see page 15 of [15] or page 39
of [26]). Nonnegativity of ¯p ∗ is a consequence of ¯p(0) = 0 and its lower
semicontinuity and convexity are satisfied because it is a convex conjugate.
Let us thus prove the upper bound.
The proof is similar to ones we have already seen a few times. Let δ > 0
and c < infF ¯p ∗ . For each x ∈ F , there is y ∈ Y such that 〈x, y〉 − ¯p(y) > c.
Let Bx = {u ∈ X : |〈u, y〉 − 〈x, y〉| < δ}, an open neighborhood of x. Since
F is compact, cover it with Bx1 , . . . , Bxm, with corresponding y1, . . . , ym.
Write
µn(Bxi ) =
and thus
Bx i
e −rn〈u,yi〉+rn〈u,yi〉
µn(du) ≤ e rn(−〈xi,yi〉+δ)
Bx i
1 lim
n→∞ rn log µn(Bxi ) ≤ −〈xi, yi〉 + δ + ¯p(yi) ≤ −c + δ.
e rn〈u,yi〉 µn(du)

50 5. Convex analysis in large deviation theory
Consequently,
lim
n→∞
1
rn log µn(F
) ≤ max
1≤i≤m
lim
n→∞
1 µn(Bxi ) ≤ −c + δ.
rn
Now take δ to 0 and c to infF ¯p ∗ .
If an LDP holds, Varadhan’s theorem (page 32) gives a sufficient condition
for the existence of the limit
1
p(y) = lim
n→∞ rn log
e rn〈x,y〉 (5.1)
µn(dx).
This function is sometimes called the pressure. By the previous theorem,
p ∗ is a candidate for the rate function. So when is it the case that p ∗ = I?
The next theorem says that this holds when I is convex.
Theorem 5.25. Assume LDP(µn, rn, I) holds on E and I is convex. If
we extend I to X by setting it to ∞ outside E, then I is convex, lower
semicontinuous on X , and LDP(µ, rn, I) holds on X . If, moreover,
(5.2)
sup
n
E
e rn〈x,y〉 µn(dx)
1/rn
< ∞ for all y ∈ Y,
then the limit in (5.1) exists, for all y ∈ Y, and is equal to
p(y) = sup{〈x,
y〉 − I(x)}.
x∈E
For the extended I, we have p = I ∗ and I = p ∗ .
∗ Exercise 5.26. Prove the theorem.
Hint: Use the contraction principle and Varadhan’s theorem.
The above theorem gives us a convex analytic characterization of the
zeroes of the rate function:
I(x) = 0 ⇐⇒ p ∗ (x) = 0 ⇐⇒ p ∗ (x) + p(0) = 0 = 〈x, 0〉 ⇐⇒ x ∈ ∂p(0).
In other words, the set of zeroes is the subdifferential of the pressure at 0.
We have already seen in Section 4.3 (Exercise 4.17(c)) that rate functions
do not have to be convex. Here is an exercise showing how easy it is to
construct such rate functions.
Exercise 5.27. Let µn and νn be probability measures on X such that
LDP(µn, rn, I) and LDP(νn, rn, J) hold. Prove that for any α ∈ (0, 1),
LDP(αµn + (1 − α)νn, rn, min(I, J)) holds. Observe that the new rate function
may fail to be convex even if I and J are convex.
The next exercise observes that superexponential convergence in the
LDP corresponds to a linear pressure functional.

5.3. Multidimensional Cramér theorem revisited 51
Exercise 5.28. Let {ηn} be R d -valued random variables and m ∈ R d . Assume
{ηn} are uniformly bounded. Prove that the following are equivalent.
(a) ∀ε > 0, limn→∞ r −1
n log P {|ηn − m| ≥ ε} = −∞.
(b) ∀t ∈ R d , p(t) = limn→∞ r −1
n log E[e rnt·ηn ] = t · m.
(c) LDP(P {ηn ∈ ·}, rn, I) holds with I(m) = 0 and I(x) = ∞ for
x = m.
Note that boundedness is only needed for (c)⇒(b).
Hint: To prove that (b) implies (a) find finitely many t1, . . . , tk ∈ R d and
δ > 0 such that {x : |x − m| ≥ ε} ⊂ ∪ s i=1 {x : x · ti ≥ ci} and m · ti + δ ≤ ci
for all i.
If (5.2) is violated, then even when all limits exist, Varadhan’s theorem
and the convex conjugate representation of the rate may fail. Consequently,
the general upper bound may not be optimal.
Exercise 5.29. (page 123 of [8]) Let 0 < pn < 1/2, bn and arbitrary sequence
of real numbers, and µn ∈ M1(R) such that µn{0} = 1 − 2pn and
µn{bn} = µn{−bn} = pn. Prove the following.
(a) If n −1 log pn → −∞, then LDP(µn, n, I) holds with the proper
convex lower semicontinuous rate I such that I(0) = 0 and I(x) =
∞ for x = 0.
(b) If furthermore bn = n−1 log pn, then (5.2) does not hold and
p(t) = lim
n→∞ n−1
log e ntx µn(dx)
exists for t ∈ R, but p ∗ (x) < I(x) for all x = 0, and consequently
p ∗ = I and p = I ∗ .
5.3. Multidimensional Cramér theorem revisited
In this section we establish the final form of Cramér’s theorem in R d . Convexity
figures prominently in the proof. Let us recall the setting. X and
{Xn} are i.i.d. R d -valued random variables, Sn = X1 + · · · + Xn, and
µn(B) = P {Sn/n ∈ B} is the law of the sample mean. The moment generating
function is M(θ) = E[e θ·X ], and
p(θ) = log M(θ)
is a (−∞, ∞]-valued convex, lower semicontinuous function on R d . Lower
semicontinuity comes from Fatou’s lemma (see, e.g. page 16 of [15] or page
45 of [26]).

52 5. Convex analysis in large deviation theory
Cramér’s theorem on R d . Assume M(θ) < ∞ in an open neighborhood of
the origin. Then LDP(µn, n, I) holds with convex, tight rate function I = p ∗ .
Remark 5.30. With more work one can show that weak LDP(µn, n, p ∗ )
holds even without the finiteness assumption on M (Corollary 6.1.6 of [8]).
Proof. This proof gives us the opportunity to display another method for
obtaining LDPs, namely subadditivity.
Step 1. We show that there exists a convex, tight rate function I such
that LDP(µn, n, I) holds.
This is where subadditivity comes in. We claim that for each open ball
B ⊂ R d , the limit
j(B) = − lim
n→∞ n−1 (5.3)
log P {Sn/n ∈ B} ∈ [0, ∞]
exists. Any convex open set would in fact work but we need only balls. Let
an = − log P {Sn/n ∈ B}. We establish two properties for this sequence,
namely
(5.4)
and subadditivity
(5.5)
either an = ∞ ∀n, or ∃N < ∞ such that an < ∞ ∀n ≥ N
am+n ≤ am + an.
To check (5.4), suppose there exists k such that P {Sk/k ∈ B} > 0. This
k is kept fixed while we verify (5.4). Since B is the union of countably many
closed balls, we can find a closed ball K ⊂ B such that P {Sk/k ∈ K} > 0.
Fix ε > 0 smaller than the distance from K to the complement of B so that
x ∈ K and |y| < ε imply x + y ∈ B.
For this proof it is convenient to use the abbreviation Sm,n = Xm+1 +
Xm+2 + · · · + Xn. Note that Sm,n has the same distribution as Sn−m.
Decompose any n as n = mk + ℓ with 0 ≤ ℓ < k, and then write
Smk+ℓ = S0,k + Sk,2k + · · · + S (m−1)k,mk + Smk,mk+ℓ.
The terms above are independent and the first m also identically distributed.
Note also the inequality
Sn
Smk
−
n mk =
ℓSmk
Smk,mk+ℓ
−
|Smk| |Smk,mk+ℓ|
n n ≤ + .
mn n
∗ Exercise 5.31. Apply an exponential Chebyshev inequality together with
the assumption that M(θ) < ∞ in a neighborhood of the origin to show
that there exists N0 such that
max
0≤ℓ

5.3. Multidimensional Cramér theorem revisited 53
Imagine a large enough radius r such that K ⊂ {x : |x| < r}. Then
(mk) −1 Smk ∈ K implies (mk) −1 Smk < nε/(2k) if n > 2kr/ε. Fix any
N > N0 ∨ (2kr/ε). Now we estimate the probability from below. First by
the choice of ε, and then by independence,
Smk Sn
Smk
P {Sn/n ∈ B} ≥ P ∈ K,
mk −
n mk < ε
Smk |Smk| ε |Smk,mk+ℓ|
≥ P ∈ K, < , <
mk mn 2 n
ε
2
Smk |Smk| nε |Sℓ| ε
= P ∈ K, < P <
mk mk 2k n 2
Smk
≥ P ∈ K ·
mk 1
for n ≥ N.
2
To prove (5.4) we use convexity of K and the i.i.d. assumption. Let now
n ≥ N.
P {Sn/n ∈ B} ≥ 1
2 · P {(mk)−1 Smk ∈ K}
≥ 1
2 · P k −1 S0,k ∈ K, k −1 Sk,2k ∈ K, . . . , k −1 S (m−1)k,mk ∈ K}
= 1
2 · P {k −1 Sk ∈ K} m > 0.
Property (5.4) has been verified.
Similarly, (5.5) follows immediately from convexity of B and the i.i.d.
assumption:
am+n = − log P {(m + n) −1 Sm+n ∈ B}
≤ − log P m −1 Sm ∈ B , n −1 Sm,m+n ∈ B
= − log P {Sm/m ∈ B} − log P {Sn/n ∈ B}
= am + an.
Note that the inequalities above work even if some probabilities vanish.
With these properties checked we can apply the next classic fact. In
Section 7.2 we generalize it suitably to a multidimensional index set.
Fekete’s lemma. Let (an)n≥1 be a sequence in (−∞, ∞] with properties
(5.4)–(5.5). Then
an
lim
n→∞ n
= inf
n
an
n
with values in [−∞, ∞].
Proof. For the identically infinite case the claimed property is trivially true.
So only the case where the sequence is eventually finite needs proof. The
inequality
an
lim
n→∞ n
≥ inf
n
an
n

54 5. Convex analysis in large deviation theory
needs no proof.
Fix any k ∈ N such that ak < ∞. Fix m0 such that m0k ≥ N for N
from assumption (5.4). Consider n > m0k and write n = mk + m0k + ℓ for
some 0 ≤ ℓ < 0. Subadditivity gives
an ≤ mak + am0k+ℓ ≤ mak + max
j:m0k≤j

5.3. Multidimensional Cramér theorem revisited 55
(5.7)
(5.8)
From Lemma 4.7 we get one inequality for free:
p(θ) ≥ I ∗ (θ).
For the trickier half observe that for an open ball B and any finite n,
n −1 log P {Sn/n ∈ B} ≤ −j(B) ≤ − inf
x∈ ¯ I(x).
B
The first inequality is the subadditivity. In the second ¯ B is the compact
closure of B, and the inequality comes from compactness. Take a number
c < inf x∈ ¯ B I(x). For each x ∈ ¯ B pick an open ball Gx ∋ x such that
j(Gx) > c. Cover ¯ B with finitely many of these balls, say G1, . . . , Gm. The
simple union bound
P {Sn/n ∈ B} ≤
m
P {Sn/n ∈ Gi}
i=1
becomes in the limit j(B) ≥ mini j(Gi) > c.
Let us first restrict the calculation of p(θ) to a δ-ball around a point
z ∈ R d and develop an estimate for it.
n −1 log E e θ·Sn , n −1 Sn ∈ B(z, δ)
≤ θ · z + |θ| δ + n −1 log P {Sn/n ∈ B(z, δ)}
≤ θ · z + |θ| δ − inf
x∈B(z,δ)
I(x)
≤ sup
x∈B(z,δ)
{θ · x − I(x)} + 2 |θ| δ.
Next we go from small balls to a large ball of radius R centered at the origin.
We cover the R-ball with m δ-balls B(zi, δ). There is a constant C such that
m ≤ (CR/δ) d for all R > 0. Then
≤ R
Define
n −1 log E e θ·Sn , n −1 Sn
≤ n −1 d log(CR/δ) + max
1≤i≤m n−1 log E e θ·Sn , n −1 Sn ∈ B(zi, δ)
≤ n −1 d log(CR/δ) + I ∗ (θ) + 2 |θ| δ.
p(θ, R) = lim
n→∞ n−1 log E e θ·Sn , n −1
Sn
≤ R .
Then from above we have the bound
p(θ, R) ≤ I ∗ (θ) + 2 |θ| δ.

56 5. Convex analysis in large deviation theory
It remains to argue that p(θ, R) is a good approximation to the true p(θ).
Independence makes this easy:
n −1 log E e θ·Sn , n −1
Sn
≤ R
≥ n −1 log E e θ·Sn , |Xi| ≤ R ∀i ∈ {1, . . . , n}
= n −1
n
θ·Xi log E e 1I{|Xi| ≤ R}
= log E e θ·X , |X| ≤ R .
i=1
Combining the last two displays gives
log E e θ·X , |X| ≤ R ≤ I ∗ (θ) + 2 |θ| δ.
Take δ ↘ 0 and R ↗ ∞ to get p(θ) ≤ I ∗ (θ). Together with (5.7) we
have p = I ∗ . Since I is convex, lower semicontinuous, and not identically
infinite (the upper bound guarantees that a rate function has inf I = 0),
I = I ∗∗ = p ∗ . This completes the proof of Cramér’s Theorem.
Let us observe that Step 2 is not needed if we assume M(θ) < ∞ for all
θ ∈ R d . For in this case I = p ∗ follows from Theorem 5.25. Furnishing the
details for this is a good exercise for the reader who sees this material for
the first time.
Exercise 5.32. A process {Xn} is exchangeable if its distribution is invariant
under finite permutations of the random variables. According to de
Finetti’s theorem (page 174), an infinite exchangeable sequence is a mixture
of i.i.d. sequences. Use this, Cramér’s theorem and either Bryc’s theorem or
Exercise 5.27 to derive an LDP for the sample mean Sn/n of an exchangeable
process under some suitable hypotheses.

Relative entropy and
large deviations for
empirical measures
Chapter 6
Exercise 4.4 showed that if {Xk} are i.i.d. Bernoulli random variables, then
the laws of the empirical measures
Ln = 1
n
satisfy a large deviation principle with the rate function H : M1([0, 1]) →
[0, ∞] given by
(6.1)
n
k=1
δXk
H(α) = Ip(s) for α = sδ1 + (1 − s)δ0 with s ∈ [0, 1],
where Ip(s) = s log s
1−s
p + (1 − s) log 1−p . Recall also the direct relation between
Ip and both thermodynamic and information-theoretic entropies; see
Sections 1.2 and 1.1.
In this chapter, we will introduce the entropy of one measure relative to
another, study its properties, and see how it is related to large deviations
for empirical measures. The above H will then be seen to be the entropy of
α relative to BER(p).
Once one has a large deviation principle for empirical measures, one can
use the contraction principle to recover a large deviation principle for the
sample mean. Hence, the former has more information about the process
than the latter.
57

58 6. Relative entropy and large deviations for empirical measures
6.1. Relative entropy
Let M(X ) be the space of finite signed measures on a measurable space
(X , B) and let M1(X ) be the subspace of probability measures. Let bB
be the space of bounded measurable functions. There is a natural duality
between M(X ) and bB:
〈ν, g〉 = g dν, for g ∈ bB and ν ∈ M(X ).
The resulting weak topologies separate points.
∗ Exercise 6.1. Show that for each g ∈ bB such that g ≡ 0 there exists a
ν ∈ M1(X ) such that 〈ν, g〉 = 0. Then show that for each ν ∈ M(X ) such
that ν ≡ 0 there exists a g ∈ bB such that 〈ν, g〉 = 0. Assuming that X is a
metric space and B its Borel σ-algebra, show that for each ν ∈ M(X ) such
that ν ≡ 0 there exists a g ∈ Cb(X ) such that 〈ν, g〉 = 0.
Definition 6.2. For ν, λ ∈ M1(X ), the entropy of ν relative to λ, H(ν | λ),
is defined by
dν
φ log φ dλ if ν ≪ λ and φ =
H(ν | λ) =
dλ ,
∞ otherwise.
Note that x log x ≥ −1/e, so the above integral is well-defined.
Exercise 6.3. Show that the entropy of a measure α ∈ M1([0, 1]) relative
to BER(p) is given by (6.1).
Lemma 6.4. H(ν | λ) ≥ 0 and H(ν | λ) = 0 if and only if ν = λ.
Proof. Jensen’s inequality (page 14 of [15] or page 40 of [26]) implies that
H is nonnegative. The strict convexity of x log x implies that equality holds
if and only if φ is constant λ-a.s.
Let us fix the reference probability measure λ ∈ M1(X ). Let p : bB → R
be defined by
p(g) = log e g dλ.
Extend H(ν | λ) to all ν ∈ M(X ) by setting it to infinity for ν ∈ M(X )
M1(X ).
Theorem 6.5. p and H are convex conjugates of one another. In particular,
H is convex and has the variation representation
(6.2)
H(ν | λ) = sup {E
g∈bB
ν [g] − log E λ [e g ]}.

6.1. Relative entropy 59
Proof. We start by proving that p = H ∗ . To this end, put dν = eg
E λ [e g ] dλ,
and compute
〈ν, g〉 − H(ν | λ) = Eλ [ge g ]
E λ [e g ]
− Eλ
= log E λ [e g ] = p(g).
e g
E λ [e g ] (g − log Eλ [e g ])
So H ∗ (g) ≥ p(g). On the other hand, let ν ∈ M1(X ) with ν ≪ λ and
φ = dν
dλ such that Eν [| log φ|] < ∞. Write
〈ν, g〉 − H(ν | λ) = E ν [g − log φ] = E ν [log eg
φ ] ≤ log Eν [ eg
φ ]
= log E λ [e g 1I{φ > 0}] ≤ log E λ [e g ] = p(g).
So H ∗ (g) ≤ p(g). Next, we show that H ≤ p ∗ .
Let ν ∈ M(X ). Suppose there exists a nonnegative g ∈ bB such that
E ν [g] < 0. Then,
p ∗ (g) ≥ E ν [−Mg] − log E λ [e −Mg ] ≥ −ME ν [g].
Taking M to infinity shows that p ∗ (g) = ∞. In the rest, we assume ν is a
nonnegative measure.
Suppose that ν(X ) > 1. Then
p ∗ (ν) ≥ E ν [M] − log E λ [e M ] = M(ν(X ) − 1)
and once again taking M to infinity shows that p ∗ (ν) = ∞. Similarly, if
ν(X ) < 1, then p ∗ (ν) = ∞. Thus, we assume ν ∈ M1(X ).
Suppose there exists a measurable set A such that λ(A) = 0 < ν(A).
Then
p ∗ (ν) ≥ E ν [M1IA] − log E λ [e M1IA ] = Mν(A)
and p ∗ (ν) = ∞. We thus assume ν ≪ λ.
Suppose φ = dν
dλ and let φb a = a ∨ (φ ∧ b), for 0 < a < 1 < b < ∞. Then
p ∗ (ν) ≥ E ν [log φ b a] − log E λ [φ b a] ≥ E ν [log φ b a] − log E λ [φ ∨ a].
Since log φ b a ≥ log a, monotone convergence (see page 16 of [15] or page 46
of [26]) implies that E ν [log φ b a] converges, as b → ∞, to E ν [log(φ ∨ a)] ≥
E ν [log φ] = H(ν | λ). Thus,
p ∗ (ν) ≥ H(ν | λ) − log E λ [φ ∨ a].
Since 0 ≤ φ ∨ a ≤ φ + 1, dominated convergence (see page 16 of [15] or page
46 of [26]) implies that log E λ [φ ∨ a] converges to log E λ [φ] = 0, as a → 0,
and we have proved that p ∗ ≥ H.
Finally, H ≤ p ∗ = H ∗∗ ≤ H, where the last inequality follows from part
(b) of Proposition 5.12.
In fact, relative entropy is strictly convex.

60 6. Relative entropy and large deviations for empirical measures
∗ Exercise 6.6. Show that if µ, ν ∈ M1(X ) and θ ∈ (0, 1), then
H(θν + (1 − θ)ν | λ) = θH(µ | λ) + (1 − θ)H(ν | λ) < ∞
is equivalent to µ = ν.
When X is a metric space Cb(X ) is also in duality with M(X ) and, by
Exercise 6.1, the topologies separate points. Restricted to M1(X ) the topology
σ(M(X ), Cb(X )) is the standard weak topology of probability measures
and is very convenient to work with; see Appendix A.1. In this case, one
can compute H using only bounded continuous functions.
Theorem 6.7. Let X be a metric space. Then
(6.3)
H(ν | λ) = sup
f∈Cb(X )
{E ν [f] − log E λ [e f ]}.
In particular, p and H are convex conjugates in the duality of the spaces
M(X ) and Cb(X ). On the space M1(X ) of probability measures H is lower
semicontinuous in the weak topology generated by Cb(X ).
The proof requires the following technical lemma which we prove in
Appendix B.1.
Lemma B.1. Let X be a metric space and let H be a class of bounded
functions that contains the space Ub(X ) of bounded uniformly continuous
functions and is closed under uniformly bounded pointwise convergence (i.e.
fn ∈ H for all n, maxn sup x |fn(x)| < ∞, and fn(x) → f(x) for all x ∈ X
together imply f ∈ H). Then bB ⊂ H.
Proof of Theorem 6.7. Let
C = sup
f∈Cb(X )
{E ν [f] − log E λ [e f ]}
and suppose C < H(ν | λ). Set H = {f ∈ bB : E ν [f]−p(f) ≤ C}. This class
of bounded functions is closed under uniformly bounded pointwise convergence
and contains Cb(X ). It thus contains all of bB and Theorem 6.5 implies
H(ν | λ) ≤ C. We thus conclude that H = p ∗ in the σ(M(X ), Cb(X ))topology.
The fact that p(f) = H ∗ (f) for f ∈ Cb(X ) is a special case of
Theorem 6.5.
We will see later how relative entropy H is related to rate functions. A
natural question to ask, then, is whether or not sublevel sets are compact.
The answer is yes, if X is Polish.
Definition 6.8. X is called Polish if it has a countable dense set and the
topology of X has a complete metric.

6.1. Relative entropy 61
Proposition 6.9. Assume X is Polish. For c ∈ R, the sublevel set
A = {ν ∈ M1(X ) : H(ν | λ) ≤ c}
is compact in the weak topology generated by Cb(X ).
To prove the lemma we need to recall the notion of tightness of a family
of measures; see Appendix A.1. We also need to recall the definition of
uniform integrability.
Definition 6.10. A collection of integrable functions C is uniformly integrable
if
lim
M→∞ sup E[|ϕ|1I{|ϕ| ≥ M}] = 0.
ϕ∈C
∗ Exercise 6.11. Prove that if C is uniformly integrable, then for all ε > 0
there exists a δ > 0 such that P (B) < δ implies sup ϕ∈C E[|ϕ|1IB] < ε.
∗ Exercise 6.12. Assume there exists a nonnegative function G : [0, ∞) →
[0, ∞) such that G(x)/x → ∞ as x → ∞, and sup ϕ∈C E[G(|ϕ|)] < ∞. Show
that C is uniformly integrable.
Proof of Proposition 6.9. Take G(x) = x log x. The criterion in the
above exercise implies that C = { dν
dµ : ν ∈ A} is uniformly integrable. Next,
we show that A is tight.
Fix ε > 0. By Exercise 6.11, there exists δ > 0 such that λ(E) < δ
implies sup ν∈A ν(E) < ε. Now use the regularity of probability measures
on Polish spaces (Ulam’s theorem; see Exercise A.11) to find a compact set
K ⊂ X such that λ(K c ) < δ. This proves the tightness of the family of
measures A.
By Prohorov’s theorem (page 169), A is relatively compact in the weak
topology. Since H is lower semicontinuous, A is closed and is thus compact.
We conclude this section with some exercises.
Exercise 6.13. For 0 ≤ f ∈ L1 (R) such that f(x) dx = 1, let
H(f) = f log f dx.
(a) Show that γ(x) = (2πσ 2 ) −1/2 e −x2 /2σ 2
is the unique minimizer of
H(f) among f that satisfy x2f(x) dx = σ2 .
(b) Similarly, show that e(x) = λe −λx is the unique minimizer among
f supported on [0, ∞) and satisfying xf(x) dx = 1/λ.
Hint: Compute H(f(x)dx | γ(x)dx) and H(f(x)dx | e(x)dx) then H(γ) and
H(e). Note that the former two quantities are nonnegative.

62 6. Relative entropy and large deviations for empirical measures
∗Exercise 6.14. (Conditional entropy formula) Suppose A is a sub-σalgebra
of B. Suppose there exist versions of the conditional probabilities
µ A and λA of µ and λ, given A; see Example 8.5. Let µA and λA be the
restrictions of µ and λ to A. Prove that
H(µ | λ) = H(µA | λA) + H(µ A | λ A )dµ.
Hint: Show that dµA
dλA = Eλ [ dµ
dλ
dµA
|A] and dλA = dµ
dλ
/ dµA
dλA .
∗ Exercise 6.15. Suppose µ(A) = 1 and λ(A) > 0. Let π = λ( · | A). Show
that
H(µ | λ) = H(µ | π) − log λ(A).
Exercise 6.16. Let S be a finite space. Show that there is a maximizing
function f in (6.2) if and only if ν and λ have the same support.
∗ Exercise 6.17. This exercise uses relative entropy to prove the Markov
chain convergence theorem for an irreducible, aperiodic, finite state space
Markov chain. Let S be a finite state space, P the transition matrix, π the
unique invariant distribution, and µ an arbitrary initial distribution. We
show that µP n → π.
(a) Show that H(· | π) is continuous.
(b) Show that in general H(νP | π) ≤ H(ν | π). Show that there exists
m ∈ N such that P m > 0 elementwise, and then H(νP m | π) <
H(ν | π) if ν = π.
(c) Let ν be a limit point of µP n . Show that ν must equal π.
6.2. Sanov’s theorem
We are now ready to prove the large deviation principle for empirical measures
of a sequence of i.i.d. random variables on a Polish space S. The
topology that determines open and closed sets in M1(S) is the weak topology
generated by Cb(S).
Sanov’s theorem. Let S be a Polish space. Let {Xn} be a sequence of i.i.d.
S-valued random variables with common law λ ∈ X = M1(S). Let
Ln = 1
n
δXk ∈ M1(S)
n
k=1
be the empirical measures. Let Qn ∈ M1(X ) be the law of Ln. Then, the
family {Qn} is exponentially tight and LDP(Qn, n, H) holds with the tight
convex rate function H(ν) = H(ν | λ).

6.2. Sanov’s theorem 63
To understand what the above theorem says let S = R and assume λ
is absolutely continuous relative to the Lebesgue measure on R; i.e. λ has a
probability density function f(x) such that λ(B) =
B f(x)dx. If one plots
the normalized histogram of data points (Xk) n k=1 , it is supposed to resemble
f. Sanov’s theorem tells us that given the null hypothesis that our data is
i.i.d. with law λ the probability that the normalized histogram will look like
the probability density function of a different measure ν decays like e−cn ,
where c is the entropy of ν relative to λ. This is what we saw in the case of
λ=BER(p) in Exercise 4.4.
Proof. Let P = λ ⊗N . Throughout the proof we have to consider expectations
with respect to more than one measure, hence now we write
E P [f] = f dP for expectation under P .
Observe that for a bounded continuous function f : S → R,
1
¯p(f) = lim
n→∞ n log
e n〈ν,f〉
1
Qn(dν) = lim
n→∞ n log EP e P
n
k=1 f(Xk)
= lim
n→∞ log
e f dλ = p(f).
Since p∗ = H, the general upper bound in Theorem 5.24 implies the upper
bound for compact sets. By Theorem 3.3, to get an upper bound for general
closed sets it is enough to establish exponential tightness of {Qn}. Use again
Ulam’s theorem (Exercise A.11) to pick a compact set Γℓ such that λ(Γc ℓ ) <
e−2ℓ2. Then, Aℓ = {ν : ν(Γℓ) ≥ 1 − 1/ℓ} is closed in the weak topology
σ(M1(S), Cb(S)) because (c) of the portmanteau theorem (Exercise A.3)
implies
ν(Γℓ) ≥ lim νj(Γℓ), if νj → ν.
j→∞
The set KL = ∩ℓ≥LAℓ is thus also closed. Since KL is also tight it is compact
(by Prohorov’s theorem, page 169). But now one has
n
Thus,
Qn(A c ℓ ) = P {Ln(Γ c ℓ ) > 1/ℓ} = P
1IΓ c ℓ (Xi)
> n/ℓ
i=1
≤ e −2nℓ
P
E e 2ℓ2 Pn i=1 1I
Γc (Xi)
= e −2nℓ E λ
e 2ℓ21I Γc (X1) n ≤ e −2nℓ (e 2ℓ2
e −2ℓ2
+ 1) n ≤ e −nℓ .
Qn(K c L) ≤
ℓ≥L
e −nℓ ≤ 2e −nL
and thereby we have verified exponential tightness of {Qn}.
Next, we focus our attention on the lower bound for open sets. This is
done using a quite standard change of measure argument. Basically, getting

64 6. Relative entropy and large deviations for empirical measures
a lower bound amounts to computing the “cost” of forcing our process to
behave differently. This means the marginal λ is replaced by some other
marginal µ. Mathematically, replacing one measure by another is done via
Radon-Nikodym derivatives, and this way relative entropy enters the calculation.
Let us now be precise.
Let G be an open subset of M1(S) and µ ∈ G. Without loss of generality,
we can assume H(µ | λ) < ∞ (otherwise, µ does not matter when computing
infG H). This implies that µ ≪ λ. Let φ = dµ
dλ and let Q = µ⊗N be the law
of the i.i.d. sequence with marginal µ. Let Fn = σ(X1, . . . , Xn). Then,
dQ
dP
(x1, . . . , xn) =
Fn
n
φ(xi) = Φn(x).
i=1
Here, we used the notation x = (xi)i≥1. Now write
1
n log P {Ln ∈ G} ≥ 1
n log EP [1IG(Ln)1I{Φn > 0}]
= 1
n log EQ [1IG(Ln)Φ −1
n ]
= 1
n log
1
Q{Ln ∈ G} EQ [1IG(Ln)Φ −1
n ] + 1
n log Q{Ln ∈ G}
−1
≥
nQ{Ln ∈ G} EQ [1IG(Ln) log Φn] + 1
n log Q{Ln ∈ G} .
In the third line above, we used Jensen’s inequality (page 14 of [15] or
page 40 of [26]) with the convex function − log x. Now use the fact that
x log x ≥ −1/e to write
Thus,
E Q [1IG(Ln) log Φn] = E Q [log Φn] − E Q [1IG c(Ln) log Φn]
1
n log P {Ln ∈ G} ≥
1
Q{Ln ∈ G}
= nE µ [log φ] − E P [1IG c(Ln)Φn log Φn]
≤ nH(µ | λ) + 1/e.
{−H(µ | λ) − 1/(ne)} + 1
n log Q{Ln ∈ G}.
By part (a) of Proposition 5.5 one can find a neighborhood of µ inside
G that is determined only by finitely many functions in Cb(S). By the law
of large numbers Q{|E Ln [f] − E µ [f]| < ε} → 1 for all such functions f and
any ε > 0. Hence, Q{Ln ∈ G} converges to 1. We thus have
1
lim
n→∞ n log Qn(G) ≥ −H(µ | λ).
Taking sup over µ ∈ G finishes the proof.

6.2. Sanov’s theorem 65
As we mentioned earlier, empirical measures contain more information
on the process than sample means do. Using the contraction principle one
can deduce a version of Cramér’s theorem for additive functionals of i.i.d.
random variables on a Polish space.
∗Exercise 6.18. Let S be a Polish space. Let {Xn} be a sequence of i.i.d.
S-valued random variables with common law λ and let φ : S → Rd be a
continuous function. Assume that E[ea|φ(X1)| ] < ∞ for all a > 0. Let µn
be the law of the sample mean Sn/n = n−1 n k=1 φ(Xk). Prove, without
applying Cramér’s theorem, that LDP(µn, n, I) holds with the tight convex
rate function I : R d → [0, ∞) given by
I(z) = inf{H(ν | λ) : E ν [φ] = z} = sup
z · θ − log
e φ(x)·θ λ(dx) : θ ∈ R d
.
Hint: Start with φ bounded. The first formula for I is a result of the
contraction principle. To prove the second equality compute I ∗ . Next,
write φ(X) = φ(X)1I{|φ(X)| ≤ b} + φ(X)1I{|φ(X)| > b}. The unbounded
part is controlled by the fact that E[e a|φ(X)| , |φ(X)| > b] → 0 when b → ∞.
The lower bound follows immediately. If I (b) is the rate corresponding to
the bounded part, then for closed F ⊂ R d one has the upper bound
lim
n→∞ n−1 log P {Sn/n ∈ F } ≤ − lim
lim
inf
ε→0 b→∞ z∈Fε
I (b) (z),
where F ε = {z + y : z ∈ F, |y| ≤ ε}. Using the fact that E[e a|φ(X)| ] < ∞
conclude the upper bound in a similar fashion to Exercise 2.16.
Remark 6.19. It is interesting that Sanov’s theorem itself follows from a
more general version of Cramér’s theorem in Polish spaces; see Theorem
6.1.3, Corollary 6.2.3, and Lemma 6.2.6 of [8].
Looking at Cramér’s theorem as a contraction from Sanov’s theorem
gives valuable insight.
∗ Exercise 6.20. Let λ ∈ M1(S) and let {Xk} be i.i.d. with marginal λ.
Fix a bounded continuous function H : S → R. (The letter H is to suggest
Hamiltonian or energy; not to be confused with relative entropy.) By
Exercise 6.18, we know that the large deviation principle holds for the laws
of
with rate function
Hn = 1
n
n
H(Xk)
k=1
p ∗ (z) = inf{H(ν | λ) : ν ∈ M1(S), E ν [H] = z},
where p ∗ is the convex conjugate of p(t) = log E λ [e tH ] < ∞, t ∈ R.

66 6. Relative entropy and large deviations for empirical measures
(a) Prove that whenever p ∗ (z) < ∞, there is a unique νz ∈ M1(S)
such that E νz [H] = z and p ∗ (z) = H(νz | λ).
Hint: Use lower semicontinuity and compact sublevel sets (Proposition
6.9) to conclude the existence of the minimizer, and strict
convexity (Exercise 6.6) to conclude its uniqueness.
(b) For β ∈ R, define µβ ∈ M1(S) by
µβ(dx) = e−βH(x)
λ(dx).
ep(−β) (In statistical mechanics, µβ is the Gibbs measure at inverse temperature
β.) Prove that p ′ (t) = E µ−t [H],
p ′′ (t) = E µ−t [H 2 ] − E µ−t [H] 2
(the variance of H under µ−t), and limt→0 p ′ (t) = H dλ (high
temperature limit).
Define
A = λ-ess inf H and B = λ-ess sup H;
i.e. A = sup{a : λ(H < a) = 0} and B = inf{b : λ(H > b) = 0}.
Show that
lim
t→−∞ p′ (t) = A and lim p
t→∞ ′ (t) = B.
(c) Assume H is not λ-a.s. constant so that p ′′ > 0. Prove that for
z ∈ (A, B), there is a unique β = βz such that p ′ (−β) = z and
p ∗ (z) = −zβ − p(−β). Furthermore, νz = µβ. In other words, for
each energy value z ∈ (A, B), there is a unique inverse temperature
β such that E µβ[H] = z, the Gibbs measure µβ minimizes the
entropy H(ν | λ) subject to the energy constraint E ν [H] = z, and
the minimum entropy is precisely the value p ∗ (z) of the Cramér
rate for Hn.
Here is an alternative point of view to the results of the above exercise,
with a thermodynamical flavor.
∗ Exercise 6.21. Let the notation be as in the previous exercise.
(a) Set Zβ = E λ [e −βH ]. (In statistical mechanics Zβ is the partition
function at inverse temperature β.) Prove that
− log Zβ = inf βE ν [H] + H(ν | λ) : ν ∈ M1(S)
is uniquely attained at µβ. When β > 0, the above can be rewritten
as
−β −1 log Zβ = inf E ν [H] + β −1 H(ν | λ) : ν ∈ M1(S) .

6.3. Maximum entropy principle 67
Hint: Apply Jensen’s inequality (page 14 of [15] or page 40 of [26])
to the integral − log(e −βH ( dν
dλ )−1 )dν.
(b) For z ∈ (A, B), prove that
sup{−H(ν | λ) : ν ∈ M1(S), E ν [H] = z}
is uniquely attained at µβ, where β is uniquely specified by the
equation E µβ[H] = z.
Hint: Existence and uniqueness are shown as in the previous exercise.
The rest is immediate from (a).
Gibbs free energy is the amount of thermodynamic energy in a system
that can be converted into work at a constant temperature and pressure,
while thermodynamic entropy multiplied by temperature is the amount of
unusable heat the system gives up when work is applied. Thus, free energy
equals the system’s energy (or, more precisely, its enthalpy) less its
thermodynamic entropy multiplied by its temperature. (Think of a group
of toddlers you want to convince of doing some project together. You will
have to spend an unproductive amount of energy just to make them sit still,
and then there is the amount of productive energy that goes into the actual
project. The latter is the free energy and the former is the entropy times the
temperature, while the total amount of energy you and the children spent
is the enthalpy.)
Part (a) of the above exercise characterizes the Gibbs measure by a
variational principle. It is a mathematical statement of the thermodynamical
principle that “left on its own, nature tends to minimize free energy (or
work)”. (On their own children will minimize the amount of work they do.)
Part (b) is the Gibbs conditioning principle, which says that “under an
energy constraint, nature maximizes entropy”, with the understanding that
thermodynamic entropy is −H(ν | λ); i.e. it corresponds to the negative
of relative entropy. (Children will maximize disorder until they exhaust
themselves.)
6.3. Maximum entropy principle
In the setting of Sanov’s theorem, Ln → λ a.s. as n → ∞. An interesting
question is: what happens if we condition the process to behave atypically?
For example, if C is a set of probability measures whose closure does not
contain λ, then P {Ln ∈ C} → 0, but what if we condition Ln to remain
in C? With some technical assumptions we can give a precise answer: Ln
converges towards the element(s) of C that minimize H( · | λ). Here is a
precise statement.

68 6. Relative entropy and large deviations for empirical measures
Maximum entropy principle. Suppose C ⊂ X = M1(S) is closed, convex,
and satisfies
inf
ν∈C
H(ν | λ) = inf H(ν | λ) < ∞.
ν∈C ◦
Then, there is a unique ˜ν ∈ C that minimizes H( · | λ) over C. The conditioned
laws of Ln converge weakly to a point mass at ˜ν, that is,
lim
n→∞ P {Ln ∈ · | Ln ∈ C} = δ˜ν
in the weak topology of M1(X ) generated by Cb(X ). In fact, this convergence
is exponential in the sense that for any weak neighborhood U of ˜ν, there is
a b > 0 such that
P {Ln ∈ U c | Ln ∈ C} ≤ e −nb
for large enough n. Furthermore, for any fixed k, the conditioned law of Xk
converges weakly to ˜ν, that is,
for all f ∈ Cb(S).
lim
n→∞ E[f(Xk) | Ln ∈ C] = E ˜ν [f]
Remark 6.22. The “entropy” in the name of the principle is thermodynamic
entropy. Hence, minimizing H( · | λ) corresponds to maximizing entropy.
As a heuristic principle maximum entropy is used in statistics to solve
the following problem: Suppose our belief is that a random variable X of
interest obeys a distribution λ. Then we receive new information about X,
perhaps by performing an experiment, and λ is no longer compatible with
the new information. What should be our new best guess for the unknown
law of X, among the compatible laws (the set C)? The answer is the law that
is closest to λ in “entropy distance”. (Quotes are in order because relative
entropy is not a metric on probability measures.) The above theorem can
be regarded as a theoretical justification for this principle.
Proof of the maximum entropy principle. As in the above exercises,
existence and uniqueness of ˜ν follow from lower semicontinuity, compact
sublevel sets, and strict convexity of H. Next, let U be any neighborhood
of ˜ν and write
1
lim
n→∞ n log P {Ln ∈ U c | Ln ∈ C}
1
= lim
n→∞ n log P {Ln ∈ U c ∩ C} − 1
n log P {Ln
∈ C} .

6.3. Maximum entropy principle 69
The assumptions of the theorem imply that 1
n log P {Ln ∈ C} converges to
−H(˜ν | λ). Since ˜ν ∈ U c and U c is closed we have
1
lim
n→∞ n log P {Ln ∈ U c | Ln ∈ C} ≤ − inf
ν∈U c H(ν | λ) + H(˜ν | λ) < 0.
∩C
For k ≤ n,
E[f(Xk) | Ln ∈ C] = E[f(Xk)1I{Ln ∈ C}]
P {Ln ∈ C}
n 1
= E f(Xk) Ln ∈ C = E{E
n
Ln [f] | Ln ∈ C}.
k=1
Since f is bounded and continuous on S, F (ν) = fdν is a bounded continuous
function on X = M1(S). Consequently by weak convergence of the
conditional distribution of Ln, E[F (Ln) | Ln ∈ C] converges to F dδ˜ν =
F (˜ν) = E ˜ν [f].
As a striking application of the maximum entropy principle, one can see
how the Gibbs measure arises as a limit of conditional probabilities with a
clear physical meaning.
∗ Exercise 6.23. (Equivalence of ensembles) With notation as in Exercise
6.20, suppose z ∈ (A, B) so there is a unique β such that E µβ[H] = z. Prove
that for any fixed k,
lim
δ→0 lim
n→∞ P {Xk ∈ · | |Hn − z| ≤ δ} = µβ.
Hint: Let Mδ = {ν : |Eν [H] − z| ≤ δ} and m(δ) = infν∈Mδ H(ν | λ). Show
that m is a continuous, decreasing function from [0, ∞) onto [0, H(µβ | λ)].
Deduce that C = Mδ satisfies the assumptions of the maximum entropy
principle. Let νδ be the entropy minimizing measure in Mδ and show that
νδ → µβ as δ → 0.
The above says, roughly, that if a large number of particles governed by
the free measure P = λ ⊗N is constrained to have average energy Hn = z
(in a controlled experiment, say), then an individual particle obeys the
Gibbs measure µβ with the temperature 1/β adjusted to produce the correct
expected energy E µβ[H] = z. In statistical mechanics, the measure
P { · | Hn = z} is called the microcanonical ensemble, and the Gibbs measure
µβ is the canonical ensemble. These measures were introduced by Josiah
Willard Gibbs (1839 -1903) who is credited with systematizing equilibrium
statistical mechanics after the pioneering work of Maxwell and Boltzmann.
The problem of equivalence of ensembles is whether these two ensembles
give equivalent results in the infinite particle limit. This exercise gives an

70 6. Relative entropy and large deviations for empirical measures
affermative answer in a simple special case: the particles do not interact,
and we look only at the marginal distribution of a single particle.
As the last item we present an elegant classic example of equivalence of
ensembles.
Maxwell’s principle. For an integer n ≥ 1 let Xn = (X (n)
1 , . . . , X (n)
n ) ∈ Rn be uniformly distributed on the (n − 1)-dimensional sphere of radius σ √ n.
For k ≤ n let Pk,n be the distribution of (X (n)
1 , . . . , X (n)
k ). Then, for each
fixed k, as n → ∞, Pk,n converges weakly to the distribution of k i.i.d.
normals with mean 0 and variance σ2 .
Proof. Let {Zk} be an i.i.d. sequence of standard normal random variables.
For k ≤ n let
Y (n)
k =
σ √ nZk
Z2 1 + · · · + Z2 .
n
∗Exercise 6.24. Prove that Yn = (Y (n)
1 , . . . , Y (n)
n ) has the same distribution
as Xn.
By the strong law of large numbers (Z2 1 + · · · + Z2 n)/n converges almost
surely to E[Z 2 1 ] = 1. The claim follows.

Large deviations for
i.i.d. fields at the
process level
Chapter 7
In the previous chapter we saw how empirical distributions have more information
than just the sample mean. However, they still do not capture the
full amount of information that the process (Xk)k≥0 has in it. The object
that does so is the sequence
Rn = 1
n
n
δ (Xj)j≥k ∈ M1(X Z+ ).
k=0
In this chapter, we study large deviations for the law of this sequence.
We generalize the setting to a random field (Xi) i∈Z d of i.i.d. random variables
indexed by the d-dimensional cubic lattice and study the large deviations
of the corresponding random measure Rn. This is called process level large
deviations, in contrast with the position level of Sanov’s theorem. Another
terminology separates large deviations into level 1 (Cramér’s theorem), level
2 (Sanov’s theorem) and level 3 (process level) large deviations. This more
general point of view turns out to be useful for statistical mechanics models,
as we will see in the second part of the course.
7.1. Setting
We are given a Polish space X . Let d ∈ N be fixed and define the space
of configurations to be Ω = X Zd.
A generic element of Ω is a configuration
ω = (ωi) i∈Zd with coordinates ωi ∈ X . Endowed with the product topology
71

72 7. Large deviations for i.i.d. fields at the process level
Ω is Polish as well. Let F be its Borel σ-algebra. Theorem A.12 implies
that M1(Ω), with the weak topology generated by Cb(Ω), is also Polish.
Call a function f : Ω → R local if it is a function of only finitely many
coordinates ωi. Let Cb,loc(Ω) be the space of bounded continuous local functions.
∗ Exercise 7.1. Show that µk → µ in M1(Ω) if and only if all finite-
dimensional marginals converge.
Hint: Use the equivalence of (a) and (b) in the portmanteau theorem (Exercise
A.3) along with a density argument.
∗ Exercise 7.2. Prove that the topologies on M1(Ω) generated by Cb(Ω)
and Cb,loc(Ω) coincide.
Hint: On each ΩΛ, Λ finite, use the argument on page 168 to find countably
many functions that determine weak convergence. Put all these functions
together to form a metric of the type (A.1) that has only local functions.
Now use Exercise A.5 and conclude.
We are also given a probability measure λ ∈ M1(X ). Let P = λ⊗Zd be
the product measure on Ω. If we define the coordinate process (Xi) i∈Zd as
Xi(ω) = ωi, then P makes this process into an i.i.d. random field. With
our statistical mechanics applications in mind, we sometimes refer to Xi’s
as spins.
The shift group {θi : i ∈ Z d } on Ω is the group of homeomorphisms
defined by (θiω)j = ωi+j for ω ∈ Ω and i, j ∈ Z d . Let Mθ(Ω) be the space
of shift-invariant probability measures on Ω, that is
Mθ(Ω) = {µ ∈ M1(Ω) : µ ◦ θi = µ ∀i ∈ Z d }.
Let I be the σ-algebra of shift-invariant Borel sets of Ω, that is
(7.1)
I = {A ∈ F : θiA = A ∀i ∈ Z d }.
Exercise 7.3. Prove that I is a σ-algebra.
Definition 7.4. We say that µ ∈ Mθ(Ω) is ergodic if µ(A) ∈ {0, 1} for all
A ∈ I. We denote the set of ergodic probability measures by Me(Ω).
The ergodic theorem implies that a shift-invariant probability measure is
a mixture of ergodic ones. Then the extreme points of the convex set Mθ(Ω)
must be ergodic. (By definition, extreme points of Mθ(Ω) are measures
P ∈ Mθ(Ω) such that P = tP1+(1−t)P2 with t ∈ (0, 1) and P1, P2 ∈ Mθ(Ω)
imply P1 = P2 = P .) The converse is also easily seen to be true. Thus,
Me(Ω) also stands for the set of extreme points of Mθ(Ω). See Appendix
A.2 for more.

7.2. Specific relative entropy 73
Let us write i = (i1, i2, . . . , id) for points of the indexing lattice Zd , and
then a sequence of cubes Vn = {i ∈ Zd : −n < i1, . . . , id < n} whose union
exhausts Zd . The empirical fields Rn : Ω → M1(Ω) are defined by
Rn(ω) = 1
|Vn|
δθiω.
By the multidimensional ergodic theorem (page 170), E Rn(ω) [g] → E P [g]
for all g ∈ L 1 (P ) and P -a.e. ω. Our goal is to study large deviations of the
laws of Rn.
i∈Vn
For ω ∈ Ω, the periodized configuration ω (n) is defined by ω (n)
i
i ∈ Vn, and ω (n)
i
= ωi for
= ω(n)
j whenever ik = jk mod (2n − 1) for all k = 1, . . . , d.
The periodic empirical fields Rn : Ω → M1(Ω) are then defined by
Rn(ω) = 1
δθiω |Vn|
(n).
(7.2)
Due to periodization, Rn is measurable with respect to FVn, the σ-algebra
generated by {ωi : i ∈ Vn}. It is, therefore, easier to work with Rn than
with Rn, which depends on the whole configuration ω. However, Rn and Rn
come asymptotically close together.
i∈Vn
∗Exercise 7.5. Prove that for all bounded local measurable functions g
sup E
ω
e
Rn(ω) Rn(ω)
[g] − E [g] → 0.
As a consequence, Rn also converges weakly to P . By Exercise 2.17 the
large deviation principle for Rn transfers easily to one for Rn, once one has
a tight rate function.
7.2. Specific relative entropy
Just like it was the case in Sanov’s theorem, the rate function in the large
deviation principle for Rn and Rn, evaluated at the measure Q ∈ Mθ(Ω),
will be the entropy of Q relative to P . However, there is one complication.
Exercise 7.6. Prove that if Q ∈ Mθ(Ω) is absolutely continuous relative
to P ∈ Me(Ω), then Q = P .
Hint: Prove that Q ∈ Me(Ω) and use Exercise A.15.
The above is due to the fact that we now have a product space and
a product measure. To get a nontrivial entropy we take a limit of finitedimensional
entropies normalized by volume.

74 7. Large deviations for i.i.d. fields at the process level
For Q ∈ M1(Ω) and Λ ⊂ Z d , let QΛ be the restriction of Q to FΛ, the
σ-algebra generated by ωΛ = (ωi)i∈Λ. Let HΛ(Q | P ) be the entropy of QΛ
relative to PΛ. For Λ = Vn we abbreviate and write Qn and Hn.
Theorem 7.7. For each Q ∈ Mθ(Ω), the specific relative entropy
exists, and is also given by
(7.3)
1
h(Q | P ) = lim
n→∞ |Vn| Hn(Q | P )
1
h(Q | P ) = sup
Λ∈R |Λ| HΛ(Q | P ),
where R is the collection of finite rectangles in Z d .
The following simple argument is sufficiently elegant and common that
it deserves to be presented as a separate lemma.
Fekete’s lemma. Suppose that for Λ ∈ R we have aΛ ∈ [0, ∞] that satisfy
Then
(a) aΛ + a∆ ≤ aΛ∪∆ whenever Λ ∩ ∆ = ∅ and Λ ∪ ∆ ∈ R.
(b) aΛ = ai+Λ for all i ∈ Z d and Λ ∈ R.
aVn aΛ
lim = sup
n→∞ |Vn| Λ |Λ| .
Proof. Fix Λ. Let {k+Λ : k ∈ K} be a tiling of Zd by disjoint shifted copies
of Λ. Let b(n) be the number of these copies that are contained in Vn. Let
{k + Λ : k ∈ K ′ } be the copies that intersect Vn but do not lie entirely inside
Vn. Set Λn k = Vn ∩ (k + Λ). Now (a) can be applied repeatedly to get the
inequality
aVn ≥ b(n)aΛ +
≥ b(n)aΛ.
k∈K ′
Thus,
aVn
lim
n→∞ |Vn| ≥ aΛ
b(n) aΛ
lim =
n→∞ |Vn| |Λ| .
The other direction is trivial.
Proof of Theorem 7.7. We verify (a) and (b) of the above lemma for aΛ =
HΛ(Q | P ). By the variational characterization (6.3) of relative entropy,
aΛ n k
HΛ(Q | P ) = sup
f∈Cb(X Λ )
{E QΛ PΛ f
[f] − log E [e ]}.
Note that we make no notational distinction between QΛ as a measure on
the σ-algebra FΛ on the space Ω, and as a measure on the space X Λ . Let

7.2. Specific relative entropy 75
f ∈ Cb(X Λ ) and g ∈ Cb(X ∆ ). Then, f and g are independent under P ,
HΛ∪∆(Q | P ) ≥ E Q [f + g] − log E P [e f+g ]
Taking sup over such f and g gives
= (E Q [f] − log E P [e f ]) + (E Q [g] − log E P [e g ]).
HΛ∪∆(Q | P ) ≥ HΛ(Q | P ) + H∆(Q | P ).
The shift invariance part (b) follows also from the variational characterization
because for Q ∈ Mθ(Ω) QΛ and Qi+Λ coincide as measures on
X Λ .
The specific entropy has nice properties.
Proposition 7.8. h( · | P ) is affine on Mθ(Ω), lower semicontinuous, and
the sublevel sets {h ≤ c} are compact in the weak topology generated by
Cb(Ω). Furthermore, h(Q | P ) = 0 if, and only if, Q = P .
Proof. Since each HΛ is convex and lower semicontinuous their supremum
h inherits these properties. Hence, we need to show that h is also concave.
To this end, suppose Q = tQ 1 +(1−t)Q 2 . We want to show that h(Q | P ) ≥
th(Q 1 | P ) + (1 − t)h(Q 2 | P ).
Assume Qn ≪ Pn for all n. (Otherwise, h(Q | P ) = ∞ and there is
nothing to prove.) Let f 1 n = dQ1n dPn and f 2 n = dQ2n . Write
dPn
Hn(Q | P ) = E Q
log dQn
dPn
= tE Q1 1
log(tfn + (1 − t)f 2 n) + (1 − t)E Q2 1
log(tfn + (1 − t)f 2 n)
≥ tE Q1 log(tf 1 n) + (1 − t)E Q2 log((1 − t)f 2 n)
= tHn(Q 1 | P ) + (1 − t)Hn(Q 2 | P ) + t log t + (1 − t) log(1 − t).
Divide by |Vn| and let n grow to infinity.
Since the weak topology is metric, to prove compactness of sublevel sets
it is enough to show sequential compactness. Thus, fix c ∈ R and let {Q ℓ }
be a sequence such that h(Q ℓ | P ) ≤ c for all ℓ ∈ N.
Since H1 ≤ h, Proposition 6.9 implies the existence of a subsequence
{Q ℓ(j,1) }j∈N with h(Q ℓ(j,1) | P ) ≤ c and such that the marginals on X V1 converge
weakly to a probability measure ρ1 ∈ M1(X V1 ). Next, H2 ≤ |V2|h implies
the existence of a further subsequence {Q ℓ(j,2) }j∈N with h(Q ℓ(j,2) | P ) ≤
c and marginals on X V2 converging weakly to a probability measure ρ2 ∈
M1(X V2 ). Inductively, given {Q ℓ(j,n−1) }j∈N, by appeal to compact sublevel
sets of relative entropy we extract a further subsequence {Q ℓ(j,n) }j∈N of

76 7. Large deviations for i.i.d. fields at the process level
{Q ℓ(j,n−1) }j∈N for which h ≤ c continues to hold and the marginals on X Vn
converge to ρn ∈ M1(X Vn ).
Now, {Qℓ(j,j) }j∈N is a subsequence of {Qℓ(j,n) }j∈N, for any fixed n, and
thus {Q ℓ(j,j)
n }j∈N converge weakly to ρn. This kind of argument is called the
diagonal trick.
Observe that if m > n, then the restriction of ρm to X Vn is the weak limit
of the restrictions of Q ℓ(j,j)
m to Vn which are precisely Q ℓ(j,j)
n and thus have
limit ρn. In other words, {ρn} form a consistent family of finite-dimensional
distributions. Kolmogorov’s extension theorem (see page 474 of [15] or page
60 of [26]) implies then that there exists a probability measure Q ∈ M1(Ω)
such that Qn = ρn for all n. But then the finite-dimensional marginals of
{Qℓ(j,j) } converge to those of Q and thus the sequence itself converges weakly
to Q. This proves the sequential compactness (and hence the compactness)
of {h ≤ c}.
The statement about the zeroes of h follows directly from Lemma 6.4
and (7.3).
Recall that Me(Ω) is the set of extreme points of the convex set Mθ(Ω).
Because the space of finite measures is infinite-dimensional, an interesting
phenomenon happens. Me(Ω) is in fact dense in Mθ(Ω). This denseness
allows us to apply the ergodic theorem to prove the lower bound in the LDP
we are after, quite similarly to the way the ergodic theorem was used to
prove the lower bound in Sanov’s theorem.
Lemma 7.9. For Q ∈ Mθ(Ω), there are Q k ∈ Me(Ω) that converge weakly
to Q and such that h(Q k | P ) → h(Q | P ).
Proof. Let {j + Vn : j ∈ In}, In = (2n − 1)Z d , be a covering of Z d by
disjoint shifted copies of Vn. Let Q n,⊗ be the measure on Ω which makes
the σ-algebras Fj+Vn, j ∈ In, independent, but coincides with Q on each
such σ-algebra. More precisely, Q n,⊗ (dω) = ⊗j∈InQj+Vn(dωj+Vn). Then,
Q n = 1
Q
|Vn|
i∈Vn
n,⊗ ◦ θ−i
is shift invariant. We prove that this sequence satisfies the claim of the
lemma.
First, we prove the weak convergence to Q. To this end, let g be any
bounded Fm-measurable function and take n > m. Then
E Qn
[g] = 1
E
|Vn|
Qn,⊗
[g ◦ θi] + O(cm,n),
i∈Vn
i+Vm⊂Vn

7.2. Specific relative entropy 77
where cm,n = |{i ∈ Vn : i + Vm ⊂ Vn}/|Vn| → 0 as n → ∞. But if
i + Vm ⊂ Vn, then E Qn,⊗
[g ◦ θi] = E Q [g]. Thus,
E Qn
[g] = (1 − cm,n)E Q [g] + O(cm,n) −→ E
n→∞ Q [g].
Next, we show that the measures Qn are ergodic. Let A be in the shiftinvariant
σ-algebra. Then,
Q n (A) = 1
Q
|Vn|
n,⊗ (θ−iA) = Q n,⊗ (A).
i∈Vn
Exercise A.18 gives a tail measurable event B such that 0 = Q n (A∆B) ≥
1
|Vn| Qn,⊗ (A∆B). Thus, Q n,⊗ (A) = Q n,⊗ (B). By Kolmogorov’s 0-1 law
(Exercise A.17), Q n,⊗ (B) ∈ {0, 1}. The ergodicity of Q n has been checked.
Finally, we need to prove the convergence of specific entropies. By the
lower semicontinuity of h we know that h(Q | P ) ≤ lim n→∞ h(Q n | P ). To
prove the other direction we first use convexity to write
h(Q k 1
| P ) = lim
n→∞ |Vn| Hn(Q k 1 1
| P ) ≤ lim
Hn(Q
n→∞ |Vn| |Vk|
k,⊗ ◦ θ−i | P ).
Note that the shift invariance of P implies that
i∈Vk
Hn(Q k,⊗ ◦ θ−i | P ) = Hi+Vn(Q k,⊗ | P ) ≤ H W i n,k (Q k,⊗ | P ),
where W i n,k is the smallest cube containing i + Vn and made up of Vk and
disjoint shifted copies of it.
Exercise 7.10. Prove the following
(a) Let X and Y be two Polish spaces. Let α, µ ∈ M1(X ) and β, ν ∈
M1(Y). Then H(α ⊗ β | µ ⊗ ν) = H(α | µ) + H(β | ν).
(b) Let X be a Polish space and λ, µ ∈ M1(X ). Then h(µ ⊗Zd
H(µ | λ).
| λ⊗Zd) =
From (a) in the above exercise H W i n,k (Q k,⊗ | P ) = |W i n,k |
|Vk| Hk(Q | P ). Since
|W i n,k | ≤ (2n + 4k)d and |Vn| = (2n − 1) d , one has
(7.4)
h(Q k | P ) ≤ lim
n→∞
1 1
Hn(Q
|Vn| |Vk|
k,⊗ ◦ θ−i | P )
i∈Vk
≤ 1
|Vk| Hk(Q | P ) ≤ h(Q | P ).
This completes the proof of the lemma.
We end this section with an exercise that shows how specific relative
entropy can in fact be seen as a relative entropy.

78 7. Large deviations for i.i.d. fields at the process level
∗ Exercise 7.11. Suppose A1 ⊂ A2 ⊂ · · · are σ-algebras generating a σ-
algebra A . Let µ and ν be two probability measures on A . Prove that
HAn (µ | ν) increases to H(µ | ν) as n → ∞. Here HAn is relative entropy of
the restrictions to An.
Hint: Use the variational formulation of relative entropy to see that the
limit exists and equals supn HAn (µ | ν). To identify the limit as H(µ | ν),
prove a suitable analogue of Lemma B.1. Note that it is easier to solve the
exercise in the special case An = FVn than for general σ-algebras because
then one can apply Lemma B.1.
Exercise 7.12. Let Q ∈ Mθ(Ω). Order Zd lexicographically (i.e. by first
coordinate, then second, etc), and let Ui = {k ∈ Zd : k < i} be the lexicographic
past of the site i. (Of course, if d = 1 this language has a clear
meaning, but the mathematics works just as well for any dimension d.) Let
Q(· | FU0 ) be a conditional probability measure for Q, given the past of 0.
Restrict this measure to F0 to get Q0(· | FU0 ), the conditional distribution
of X0 under Q, given the past of 0. Let U ∗ 0 = U0 ∪ {0}. Think of QU0 ⊗ λ as
a probability measure on FU ∗ 0 = FU0 ⊗ F0 in the obvious way. Prove that
h(Q | P ) = H(QU ∗ 0
| QU0 ⊗ λ).
Hint: Fix Vn = {i (1) > i (2) > · · · > i (s) }, with s = |Vn|. Let V k n = {i (ℓ) :
k ≤ ℓ ≤ s}. Note that
s−1
Hn(Q | P ) = H0(Q | P ) +
k=1
{H V k n −i (k)(Q | P ) − H V k+1
n
−i (k)(Q | P )}.
By the conditional entropy formula (Exercise 6.14), the summation term
is equal to H V k n −i (k)(QU ∗ 0 | QU0 ⊗ λ). Now any fixed finite subset of U ∗ 0 is
contained in V k n − i (k) for all but an asymptotically vanishing fraction of
i (k) ∈ Vn.
7.3. Pressure and the large deviation principle
We are now ready to state the main theorem of this chapter. Throughout
this section, the setting is the one defined in Section 7.1. For Q ∈ M1(Ω)
define I(Q) = h(Q | P ) if Q ∈ Mθ(Ω), and I(Q) = ∞ otherwise.
Theorem 7.13. Let µn be the laws of the empirical fields Rn under the
i.i.d. product measure P. Then LDP(µn, |Vn|, I) holds and I is tight. The
same holds for the periodized empirical fields Rn.
The rest of this section is dedicated to the proof of this theorem. The
proof of the lower bound is very similar to those of Cramér’s and Sanov’s

7.3. Pressure and the large deviation principle 79
2m-1+2r
2m-1
Figure 7.1. Cubes V (ℓ)
m .
theorems and will use the ergodic theorem instead of the law of large numbers.
The upper bound will follow from the general Theorem 5.24 combined
with exponential tightness. The first step is to prove that the limit that
defines the pressure in (5.1) exists.
Proposition 7.14. Setting as in the statement of the theorem. Let f be
a bounded measurable local function. Then the limit defining the pressure
exists, and is the same for Rn and the periodized empirical process Rn:
p(f) = lim
n→∞
= lim
n→∞
1
log E e
|Vn| P
i∈Vn f◦θi
1
log E e
|Vn| |Vn|ERn(ω) [f]
= lim
n→∞
In particular, this holds for all f ∈ Cb,loc(Ω).
2n-1
1
log E e
|Vn| |Vn|E e
Rn(ω) [f]
.
Proof. Pick r so that f is FVr-measurable. Take two integers m < n and
let V (ℓ)
m ⊂ Vn, ℓ = 1, . . . , k d , be k d shifted copies of Vm arranged so that
there is distance 2r between each adjacent pair in each coordinate direction.
2n−1
Let k be as large as possible. In fact, k = [ 2m+2r−1 ]; see Figure 7.1.
with
The volume of Vn not covered by the copies of Vm is
|Vn| − k d |Vm| ≤ |Vn| −
n − m − r
d |Vm| = |Vn|κn,m,
m + r
lim
m→∞ lim
n→∞ κn,m
|Vm|
= lim 1 −
m→∞ (2m + 2r) d
= 0.

80 7. Large deviations for i.i.d. fields at the process level
Since f is FVr-measurable, the collections {f ◦ θi : i ∈ V (ℓ)
m } are inde-
pendent for distinct ℓ. Write
(7.5)
pn(f) = 1
log E
|Vn|
≤ 1
log E
|Vn|
e P
i∈Vn f◦θi
Pk d P
ℓ=1
e
i∈V (ℓ)
m
= κn,m f∞ + kd
log E
|Vn|
≤ κn,m f ∞ +
f◦θi
e κn,m|Vn|f
∞
e P
i∈Vm f◦θi
|Vm|
pm(f).
(2m + 2r − 1) d
Taking n → ∞ then m → ∞ one has limn→∞ pn(f) ≤ lim m→∞ pm(f).
This proves the existence of the limit p(f). To prove the second claim, use
Exercise 7.5 to write
e −|Vn|εn E
exp |Vn|E Rn(ω) [f]
≤ E
≤ E
exp |Vn|E e Rn(ω) [f]
exp |Vn|E Rn(ω) [f]
e |Vn|εn ,
where εn = sup ω |E e Rn(ω) [f] − E Rn(ω) [f]| → 0 by Exercise 7.5.
In fact, one also has an upper bound on the value of the pressure p.
Lemma 7.15. If f is a bounded FVm-measurable function, then
p(f) ≤ 1
|Vm| log E[e|Vm|f ].
Proof. Pick n so that Vn is a union of r = |Vn|/|Vm| disjoint shifted copies
of Vm. Let x1, . . . , xr be the centers of these copies, so Vn = ∪ r k=1 (xk +
Vm). Then, use the generalized version of Hölder’s inequality (proved by
induction) and then independence to write
E e P
i∈Vn f◦θi
= E e Pr k=1 f◦θx
k +j ≤
=
j∈Vm
j∈Vm k=1
j∈Vm
E e |Vm| Pr k=1 f◦θx
1/|Vm| k +j
r
E e |Vm|f 1/|Vm| |Vm|f
= E e |Vn|/|Vm|
.
The claim follows immediately.
Put the space M(Ω) of finite Borel measures in duality with Cb,loc(Ω).
We identify the convex conjugate of p in this duality. By Lemma B.1 we get
(7.6)
H(ν | λ) = sup
f∈Cb,loc(X )
{E ν [f] − log E λ [e f ]}.

7.3. Pressure and the large deviation principle 81
Lemma 7.16. For Q ∈ Mθ(Ω), p ∗ (Q) = h(Q | P ).
Proof. Let f be a bounded continuous FVm-measurable function. Then
p ∗ (Q) ≥ E Q [f/|Vm|] − p(f/|Vm|) ≥ (E Q [f] − log E[e f ])/|Vm|.
Taking sup over such f and using (7.6) implies p ∗ (Q) ≥ Hm(Q | P )/|Vm|, for
all m, and thus p ∗ (Q) ≥ h(Q | P ).
Conversely, ¯ f =
i∈Vn f ◦ θi is FVm+n-measurable, hence
Hm+n(Q | P ) ≥ E Q [ ¯
f] − log E e P
i∈Vn f◦θi
= |Vn|(E Q [f] − pn(f)),
where pn(f) was defined in (7.5). Dividing by |Vn| and taking n → ∞ one
finds that h(Q | P ) ≥ E Q [f] − p(f) and hence h(Q | P ) ≥ p ∗ (Q).
∗ Exercise 7.17. Prove that if Q ∈ M(Ω) Mθ(Ω), then p ∗ (Q) = ∞.
Next, we deal with exponential tightness.
Lemma 7.18. For all b > 0, there exists a compact set Kb ⊂ M1(Ω) such
that P { Rn ∈ Kb} ≤ e −|Vn|b .
Proof. By Sanov’s theorem (page 62), the laws of Ln = |Vn| −1
i∈Vn δXi
form an exponentially tight family of measures on M1(X ). Thus, for each
m ∈ N and j ∈ Zd , there is a compact Am,j ⊂ M1(X ) such that
P {Ln ∈ Am,j} ≤ e −|Vn|(m+|j|) .
Since Am,j is compact, Prohorov’s theorem (page 169) implies it is a tight
family of probability measures and there is a compact Um,j ⊂ X such that
µ(U c m,j ) < e−(m+|j|) for all µ ∈ Am,j. Define
Hm = {Q ∈ M1(Ω) : Q(ω : ωj ∈ U c m,j) ≤ e −(m+|j|) ∀j ∈ Z d }
and observe that if Qℓ ∈ Hm converge weakly to Q, then the portmanteau
theorem (Exercise A.3) implies
Q(ωj ∈ U c m,j) ≤ lim Qℓ(ωj ∈ U
ℓ→∞
c m,j) ≤ e −(m+|j|) .
Thus, Hm is closed. Take ℓ large enough such that
m≥ℓ−b
j e−(m+|j|) ≤ 1
and define Kb = ∩m≥ℓHm. Then, Kb is a closed tight family of measures
and hence, by Prohorov’s theorem, is compact. Indeed, for ε > 0 take m ≥ ℓ
such that
j e−(m+|j|) < ε. By Tychonov’s theorem (see Theorem A3 in
[33]), Um =
j∈Z d Um,j is compact and, for Q ∈ Kb,
Q(U c m) ≤
Q(ωj ∈ U c m,j) ≤
e −(m+|j|) < ε.
j
j

82 7. Large deviations for i.i.d. fields at the process level
Finally,
But
Figure 7.2. The bijection from j + Vn onto Vn.
P { Rn ∈ Kb} ≤
P { Rn ∈ Hm}
m≥ℓ
≤
P Rn(ωj ∈ U c m,j) > e −(m+|j|)
.
m≥ℓ
j
Rn(ωj ∈ U c m,j) = 1
δθiω |Vn|
i∈Vn
(n)(ωj ∈ U c m,j) = 1
1IU
|Vn|
i∈Vn
c m,j (ω(n)
i+j )
= 1
|Vn|
˜ι∈Vn
1IU c m,j (ω˜ι) = Ln(U c m,j),
where we observe that there is a bijection j + i ↦→ ˜ι from j + Vn onto Vn
such that ω (n)
i+j = ω˜ι; see Figure 7.2.
Since Ln(U c m,j ) > e−(m+|j|) implies Ln ∈ Ac m,j , we have
P { Rn ∈ Kb} ≤
e −|Vn|(m+|j|)
−|Vn|b
≤ e
m≥ℓ
j
m≥ℓ−b
j
e −(m+|j|) ≤ e −|Vn|b .
The lemma is proved.
We are ready to complete the proof of the process level LDP.
Proof of Theorem 7.13. Recall that the topology on M1(Ω) generated
by Cb(Ω) is the same as that generated by Cb,loc(Ω); see Exercise 7.2. The
upper large deviation bound (2.3) for compact sets, for both the laws of Rn
and of Rn, follows hence from Theorem 5.24 (with E = M1(Ω)), Proposition
7.14, Lemma 7.16, and Exercise 7.17. Then, by Theorem 3.3, exponential

7.3. Pressure and the large deviation principle 83
tightness implies the upper bound holds for all closed sets. It remains to
prove the lower bound.
By Lemma 7.9, it suffices to show that for open G ⊂ Mθ(Ω),
1
lim
n→∞ |Vn| log P {Rn ∈ G} ≥ − inf h(Q | P ).
Q∈G∩Me(Ω)
Thus let Q ∈ G be ergodic. We may assume h(Q | P ) < ∞, otherwise
there is nothing to prove. In this case we have, for all n, a Radon-Nikodym
derivative fn = dQn
dPn on Fn.
line,
Then, using Jensen’s inequality in the third
(7.7)
1
|Vn| log P {Rn ∈ G} ≥ 1
|Vn| log
= 1
|Vn| log Q{Rn ∈ G} + 1
|Vn| log
≥ 1
|Vn| log Q{Rn ∈ G} −
Now, x log x ≥ −1/e, for x > 0, to write
log fn dQn = log fn dQn −
Rn∈G
1IG(Rn)f −1
n dQn
1
1
Q{Rn ∈ G}
|Vn|Q{Rn ∈ G}
Rn∈G c
Rn∈G
Rn∈G
f −1
n dQn
log fn dQn.
fn log fn dP ≤ Hn(Q | P ) + 1/e.
Thus, (7.7) is bounded below by
1
|Vn| log Q{Rn
1 1
∈ G} −
Q{Rn ∈ G} |Vn| Hn(Q
1
| P ) −
e|Vn|Q{Rn ∈ G} .
By part (a) of Proposition 5.5 one can find a neighborhood of Q inside G
that is determined only by finitely many functions in Cb(Ω). By the ergodic
theorem Q{|E Rn [f] → E Q [f]| < ε} → 1 for all such functions f and any
ε > 0. Hence, Q{Rn ∈ G} → 1 as n → ∞ and the lower bound follows. The
lower bound also holds for Rn as a consequence of Exercises 7.5 and 2.17.
The theorem is proved.
In the next part we study Gibbs measures and their connection to large
deviation theory. We end this part of the book with an exercise on a phenomenon
related to the one observed in Section ??, this time in the context
of coin tossing.
Exercise 7.19. This exercise shows that if a coin tossing process is conditioned
to yield an abnormally large fraction of 1’s, the entire process behaves
(in the limit) like a sequence of tosses from a biased coin.
Let Ω = {0, 1} N be the sample space for coin tosses, with sample points
ω = (xk)k∈N and coordinate variables Xk(ω) = xk. For s ∈ [0, 1] let νs =
(1 − s)δ0 + sδ1 be the probability measure on {0, 1} that gives a 1 with

84 7. Large deviations for i.i.d. fields at the process level
probability s, and Qs = ν ⊗N
s
the probability measure on Ω under which the
{Xk} are i.i.d. νs-distributed. Let P = Q 1/2 be the fair coin tossing measure,
and Sn = X1 + . . . + Xn the number of 1’s in the first n observations.
Let 1/2 < s ≤ 1. Show that P { · | Sn ≥ ns} converges weakly to Qs.
Hint: To prove this you can use large deviation theory in a manner analogous
to the proof of the maximum entropy principle (page 68). Observe
that if f is a bounded local function on Ω, then
E P [f | Sn ≥ ns] = E P E Rn [f] | Sn ≥ ns + a small error.
Use Exercise 7.12 to show that Qs is the unique minimizer of h(Q | P ) subject
to E Q [X0] ≥ s.

Part II
Statistical Mechanics

Formalism for classical
lattice systems
Chapter 8
Recall the setting from the previous chapter. Let X be a Polish space. This
will be the single spin space. Ω = X Zd is the space of spin configurations.
Generic elements of Ω are primarily denoted by σ, and on occasion by ω or
τ. On Ω there is an a priori or reference measure λ that reflects complete
lack of interaction. Hence, we assume that λ is a product measure under
which the spins (σi) i∈Zd are i.i.d. X -valued random variables. For example,
if X = {−1, +1} then a natural choice is the fair coin tossing measure
λ = ( 1
⊗Zd
δ+1) .
2δ−1 + 1
2
Knowing σ ∈ Ω means we have perfect knowledge of the microscopic
state. This involves too many variables and physical measurements are subject
to statistical fluctuations. Knowing the distribution of the microscopic
states is thus more reasonable in that it leads to fewer parameters and explains
the apparent deterministic behavior of the system. We are hence
given a probability measure µ ∈ M1(Ω) that we call the macroscopic state.
8.1. Finite volume model
Fix a finite subset Λ ⊂ Z d and let ΩΛ = X Λ . Let FΛ be the corresponding
Borel σ-algebra. A Hamiltonian HΛ is a bounded FΛ-measurable function
that gives the energy HΛ(σΛ) of a configuration σΛ ∈ ΩΛ. The Gibbs measure
µΛ describes the equilibrium of the interacting spins in the volume Λ.
87

88 8. Formalism for classical lattice systems
It is justified via equivalence of ensembles (Exercise 6.23) and is defined as
(8.1)
dµΛ = e−βHΛ
dλΛ,
ZΛ
where ZΛ = E λΛ[e −βHΛ] is the normalizing constant called the partition
function, λ ∈ M1(Ω) is the a priori (or reference) measure, and β > 0 is the
inverse temperature β = 1/T .
The equilibrium of the interacting spins is also defined by a variational
principle: the free energy −β −1 log ZΛ (from statistical mechanics) minimizes
the Helmholtz free energy (from thermodynamics),
(8.2)
−β −1 log ZΛ = inf
ν∈M1(ΩΛ) {Eν [HΛ] + β −1 H(ν | λΛ)},
and the measure at which the infimum is attained is called the equilibrium
measure. These two descriptions of the equilibrium coincide; recall Exercise
6.21.
The above variational principle reflects a fundamental idea in this subject:
a balance between two competing microscopic effects. On the one
hand, the interaction of the spins, measured by the Hamiltonian, induces
an ordering effect. On the other hand, thermal motion, measured by the
entropy, has a randomizing effect. At higher temperatures, or small β, we
expect more thermal motion, while at low temperatures, or high β, we expect
more order. We have already observed such a balance in the Curie-Weiss
model in Section 4.3 where, in the absence of an external magnetic field,
magnetization occurs at low temperature and is lost at high temperature.
We wish to develop this formalism for infinite volume Λ = Z d in order
to observe phase transition. In infinite volume (8.1) and (8.2) do not work
directly. In the previous chapter we already ran into the problem that relative
entropy becomes somewhat trivial in infinite volume, and we corrected
this by taking limits normalized by volume. In a similar vein we introduce
next an interaction potential that will describe the ordering effect across microscopic
distances on the lattice. This enables us to define infinite volume
Gibbs measures.
8.2. Potentials and Hamiltonians
Definition 8.1. Φ = {ΦA : A ⊂ Z d finite, ΦA : Ω → R} is an absolutely
summable, shift-invariant interaction potential if for all i ∈ Z d and A ⊂ Z d
the following holds:
(a) ΦA is FA-measurable,
(b) ΦA+i = ΦA ◦ θi,
(c) Φ =
A:0∈A ΦA ∞ < ∞, with ΦA ∞ = sup |ΦA|.

8.2. Potentials and Hamiltonians 89
Let B be the Banach space of absolutely summable, shift-invariant interaction
potentials. The summability in part (c) of the above definition will
make all the sums we write from now on well defined.
ΦA(σ) represents the energy of the interaction of the spins in A. We will
say that it has finite range R if diam(A) > R implies ΦA ≡ 0.
Exercise 8.2. Prove that finite range shift-invariant potentials are dense in
B.
We say that Φ is a two-body or pair potential if ΦA ≡ 0 when A has
more than two points (i.e. |A| > 2). A pair potential with range 1 is called
a nearest-neighbor potential.
In what follows, we will only consider interaction potentials such that
Φ ⊂ Cb(Ω). This is necessary because the large deviation principle we proved
in the previous chapter was in the weak topology.
Note that the one-body or self-potential Φ {i}, for i ∈ Z d , does not contribute
to interaction and hence could be subsumed in the a priori measure λ.
Physically, however, the a priori measure is often canonically associated to
the single spin space X (i.e. λ is a product measure), while the self-potential
describes the effect of external forces on the spin, such as external magnetic
fields.
Let Λ ⊂ Zd be finite. The Hamiltonian Hfree Λ = HΦ,free
Λ for the volume
Λ with free boundary condition (i.e. no influence from Λc ) is the bounded
FΛ-measurable function
H free
Λ =
ΦA.
A:A⊂Λ
The interaction between spins in Λ and Λ c is described by
WΛ,Λc = W Φ Λ,Λc =
A∩Λ=∅
A∩Λc=∅ ΦA.
Now fix a configuration τΛc ∈ ΩΛc on the sites outside Λ. The Hamil-
tonian H τΛc Λ = HΦ,τ Λc Λ for Λ with boundary condition τΛc is
H τΛc Λ (σΛ) = H free
Λ (σΛ) + WΛ,Λc(σΛ, τΛc) =
A∩Λ=∅
ΦA(σΛ, τΛc). When a boundary condition τΛc has been specified, the Gibbs measure
πΛ(τ, ·) of the spins in Λ is defined in terms of the integral of a bounded test
function f on ΩΛ by
(8.3)
ΩΛ
f(σΛ) πΛ(τ, dσΛ) = 1
Z τ Λ c
Λ
ΩΛ
f(σΛ)e −βHτ Λ c
Λ (σΛ) λΛ(dσΛ).

90 8. Formalism for classical lattice systems
Alternatively, for a test function f : Ω → R,
(8.4)
Ω
f(σ) πΛ(τ, dσ) = 1
Z τ Λ c
Λ
As before, Z τΛc
Λ =
called the partition function.
ΩΛ
f(σΛ, τΛ c)e−βHτ Λ c
Λ (σΛ) λΛ(dσΛ).
ΩΛ e−βHτ Λc Λ (σΛ) λΛ(dσΛ) is the normalization constant
∗Exercise 8.3. Show that the Gibbs kernels defined by (8.4) for a potential
satisfying Definition 8.1 has the following shift invariance: for f ∈ bF on Ω,
f(θ−iσ) πΛ(τ, dσ) = f(σ) πΛ+i(θ−iτ, dσ)
Ω
Before going forward with Gibbs measures, we turn to a study of abstract
probability-measure-valued functions of the above type.
8.3. Specifications
Given two measurable spaces (Y, C ) and (Z, D), a stochastic kernel π from
(Y, C ) to (Z, D) is a map π : Y × D → [0, 1] such that
(a) π(y, ·) ∈ M1(Z, D) for all y ∈ Y,
(b) y ↦→ π(y, D) is C -measurable for all D ∈ D.
Physically, the stochastic kernel reflects the equilibrium in Z given a
y ∈ Y. From a dynamical systems point of view, the notion of a stochastic
kernel is a generalization of a dynamical system given by a map. A stochastic
kernel acts as a linear transformation that pulls functions back and pushes
measures forward. The bD → bC map f ↦→ πf is defined by
πf(y) = f(z)π(y, dz), y ∈ Y
and the M(Y, C ) → M(Z, D) map µ ↦→ µπ is defined by
µπ(D) = π(y, D)µ(dy), D ∈ D.
Y
Z
Sometimes we will write π y (D), π y f, and π(f) instead of π(y, D), πf(y),
and πf, respectively.
This action has a clear interpretation in the Markov chain setting.
Example 8.4. The transition probability P (x, dy) of a time-homogeneous
Markov chain is a stochastic kernel on the state space of the chain.
Another very basic example is conditional probability.
Ω

8.3. Specifications 91
Example 8.5. Let X be Polish and B be its Borel σ-algebra. Let A ⊂
B be a smaller σ-algebra and µ ∈ M1(X ). Then, there always exists a
conditional probability measure for µ, given A , which is a stochastic kernel
π from (X , A ) to (X , B) such that E µ [fg] = E µ [f πg] for all bounded A -
measurable functions f and all bounded B-measurable functions g. See
pages 33 and 230 of [15].
Stochastic kernels can be composed. Indeed, if π is a kernel from (Y, C )
to (Z, D) and ρ is a kernel from (Z, D) to (W, E ), then
πρ(y, E) = ρ(z, E)π(y, dz)
is a kernel from (Y, C ) to (W, E ).
Exercise 8.6. Check that the above composition does give a stochastic
kernel and that it agrees with πρf = π(ρf) and µ(πρ) = (µπ)ρ.
Going back to our original setting, we now can define what we mean by
a specification.
Definition 8.7. A specification is a family of stochastic kernels Π = {πΛ :
Λ ⊂ Z d finite} such that
(a) πΛ is a stochastic kernel from (Ω, FΛc) to (Ω, F ).
(b) πΛ is FΛ c-proper; i.e. if B is FΛ c-measurable, then πΛ(σ, B) =
1IB(σ), for all σ ∈ Ω, or πΛ1IB = 1IB.
(c) If ∆ ⊂ Λ ⊂ Z d are finite, then πΛπ∆ = πΛ.
Note that the restriction of πΛ(σ, ·) to FΛ can be physically thought of as
the equilibrium probability for the volume Λ, given the boundary condition
σΛc outside Λ.
Exercise 8.8. Let ∆ ⊂ Λ ⊂ Z d be finite. Prove that (b) implies that
π∆πΛ = πΛ.
∗ Exercise 8.9. Prove that (b) is equivalent to πΛ(fg) = gπΛf for g ∈ bFΛ c
and f ∈ bF .
Exercise 8.10. Assume that πΛf = E[f | FΛc] for some unknown probability
P . Show that condition (c) is then the usual consistency condition for
conditional expectations; i.e.
.
E[f | FΛ c] = E E[f | F∆ c] FΛ c
The physical interpretation of (c) is that equilibrium in a volume Λ is
compatible with the equilibria of all its subvolumes ∆.

92 8. Formalism for classical lattice systems
Definition 8.11. For a given specification Π, define the set
G Π = {µ ∈ M1(Ω) : E µ [f|FΛ c] = πΛf, ∀f ∈ bF , ∀Λ ⊂ Z d finite}.
This is the set of Gibbs measures of Π. It consists of probability measures
admitted by (or consistent with) Π.
A natural question to ask is whether G Π is empty or not. To answer
this question we will first re-express and weaken the criterion of belonging
to G Π .
Lemma 8.12. The following are equivalent.
(a) µ ∈ G Π ,
(b) µπΛ = µ, for all Λ ⊂ Z d finite,
(c) µπVn = µ, for all n, where Vn = {i ∈ Z d : −n < i1, . . . , id < n}.
Proof. The implication (a) ⇒ (b) follows from E µ (E µ [f|FΛ c]) = Eµ [f].
(b) trivially implies (c). To prove that (c) implies (b) suppose Λ ⊂ Vn.
Then
µπΛ(A) = µπVnπΛ(A) = µ(πVnπΛ)(A) = µπVn(A) = µ(A).
Finally, (b) implies (a) because if g ∈ bFΛc and f ∈ bF , then
Exercise 8.13. Prove that
E µ [fg] = E µ [πΛ(fg)] = E µ [g πΛf].
G Π = {µ ∈ M1(Ω) : E µ [f|FΛ c] = πΛf, ∀f ∈ Cb(Ω), ∀Λ ⊂ Z d finite}
= {µ ∈ M1(Ω) : E µ [f|FΛ c] = πΛf, ∀f ∈ Cb,loc(Ω), ∀Λ ⊂ Z d finite}.
Hint: Note that the proof of the above lemma works word for word with
these alternate definitions of G Π .
Definition 8.14. We say that the specification Π is Feller-continuous if for
all finite Λ ⊂ Z d , πΛ is a Feller-continuous kernel; i.e. πΛf ∈ Cb(Ω) for all
f ∈ Cb(Ω).
Theorem 8.15. Suppose Π is a Feller-continuous specification. Let Γn be
any sequence of finite subsets of Z d that exhausts Z d , in the sense that any
finite Λ ⊂ Z d is included in Γn for large enough n. Let νn ∈ M1(Ω) be any
sequence. If νnπΓn converge weakly to some µ ∈ M1(Ω), then µ ∈ G Π .
Proof. Fix a finite Λ ⊂ Z d and an f ∈ Cb(Ω). Then πΛf ∈ Cb(Ω) and
E µ [πΛf] = lim
n→∞ Eνn [πΓnπΛf] = lim
n→∞ Eνn [πΓnf] = E µ [f].
As a consequence of the above theorem, one has the existence of Gibbs
measures when X is compact and the specification is Feller-continuous.

8.4. Gibbs specifications and phase transition 93
8.4. Gibbs specifications and phase transition
After discussing specifications in general, let us return to specifications Π =
{πΛ : Λ ⊂ Z d finite} defined as in (8.3) by an interaction potential Φ and
a given inverse temperature parameter β > 0. This type of specification is
called a Gibbs specification. We assume our interaction potentials always
continuous: Φ ⊂ Cb(Ω). When there is no confusion, dependence on Φ is
sometimes omitted from the notation.
Exercise 8.16. Prove that if the interaction potential is continuous, then
Π = {πΛ : Λ ⊂ Z d finite} defined by (8.3) is a Feller-continuous specification.
G Π is the set of Gibbs measures for the potential Φ at inverse temperature
β. The equations µπΛ = µ defining G Π are called the DLR equations.
This idea was introduced in 1968 by Dobrushin [12] (1968) and in 1969 by
Lanford and Ruelle [28] (1969).
What makes these models interesting is that they are realistic enough to
account for the physical phenomenon known as phase transition. Physically,
a phase transition is an abrupt change in the physical properties of the
system, like water boiling into steam.
More specifically, we are given a family of absolutely summable shiftinvariant
interaction potentials {Φα} ⊂ B. As mentioned above, Φα is continuous
for all α. The index α refers to the thermodynamic variables, other
than the temperature; e.g. the external magnetic field (denoted by h) or
the pressure (denoted by P ), etc. Since all thermodynamic quantities can
be expressed in terms of the free energy and its derivatives, one way to
characterize the occurrence of a phase transition at (β, α) is by the formation
of singularities in the free energy (i.e. one of its derivatives becomes
discontinuous).
∗ Exercise 8.17. Observe that the finite volume Curie-Weiss model (Section
4.3) does not have a phase transition at any β > 0 and h ∈ R. Prove that
the infinite volume limit of the Curie-Weiss model (Theorem 4.15) exhibits
a phase transition at points (β, h) with h = 0 and β ≥ 1/J.
Hint: Use the analytic implicit function theorem (page 34 of [19]) to show
that the magnetization m(β, h) is analytic when h = 0 or β < 1/J. Also,
show that m has a jump when β > 1/J and h crosses 0 and that m(β, 0) ∼
3(Jβ − 1) as β ↘ 1/J. Next, apply Varadhan’s theorem (page 32) to find
that the limit of the free energy per spin is given by
1
−p(β, h) = − lim
n→∞ nβ log
(8.5)
e −βHn dPn = β −1 I(m) − J
2 m2 − hm.
Use the equation m solves to show that m = − ∂p
∂h , when h = 0 or β < 1/J.

94 8. Formalism for classical lattice systems
As observed in the above exercise, there are no singularities in the free
energy of a system of a finite size. The infinite volume limit, however, can
lead to singularities. The convergence to the thermodynamic limit is fast,
so that the phase behavior is apparent already on a relatively small volume,
even though the singularities are smoothed out by the system’s finite size.
Another definition of phase transition is used in the context of Gibbs
measures and was introduced by Dobrushin [12].
Definition 8.18. We say that a phase transition occurs at (β, α) when
G Πβ,α has more than one element.
In other words, phase transition is characterized by nonuniqueness of the
Gibbs measure. This is reasonable, since it means the system has several
possible equilibria to choose from. That this definition is physically sound
will be clearly illustrated in the next chapter, when we study the Ising model.
In fact, Theorem 10.2 confirms that this definition does correspond to the
physical phenomenon of phase transition. For now, however, let us just
describe things using the Curie-Weiss model even though, strictly speaking,
this model does not correspond to any Gibbs specification and thus does not
fit into the setting in this chapter.
To make our illustration even more clear, we will describe the more
familiar liquid-gas phase transition instead of the magnetization phase transition.
This is done by simply reinterpreting +1 as “the site is occupied by
a water molecule” and −1 as “the site is empty”. The empirical mean Sn/n
is then linearly related to the density of particles. A positive mean indicates
a liquid phase while a negative mean characterizes the gas phase.
The parameter β is still the inverse temperature, while the parameter h
is now called the chemical potential. The more familiar physical quantity is
pressure. The relation between pressure and chemical potential is complicated,
but increasing h corresponds to increasing the pressure. The phase
diagram in Figure 4.1 is then transformed into the one in Figure 8.1.
If looked at qualitatively (i.e. not looking at the numbers, or the precise
equation of the curve), Figure 8.1 corresponds to the phase diagram for both
the Curie-Weiss and the two-dimensional Ising models, after the chemical
potential h was replaced by the pressure. (This explains why instead of a
straight line h = 0, we get a curve.)
On the other hand, when looked at quantitatively (i.e. now take the
numbers into account), this figure corresponds to the liquid-gas part of the
experimental phase diagram for water. (We leave out the liquid-solid phase
transition, etc.) This figure shows, for example, that if pressure is held
constant and temperature is increased, water will turn into steam at a critical

8.4. Gibbs specifications and phase transition 95
Pc = 220.6
1
0.006
P (bar)
0.01
δ m, m > 0
Liquid
1
2 (δ −m + δ m ), m > 0
100
δ 0
δ m, m < 0
Gas
T c = 374
T (C)
Figure 8.1. The phase diagram for the liquid-gas phase transition in
water. T = 1/β and m = m(β, h) is as in Theorem 4.15. The solid
line represents the values of temperature and pressure at which a phase
transition occurs; i.e. to β < βc and h = 0.
temperature; e.g. if the pressure is equal to 1 bar, then water boils at the
familiar 100 ◦ C.
Now we can see how Definition 8.18 agrees with the physical phase transition.
Along the line of phase transition, the average density takes one of
two values, with equal probability. In the Ising model, this corresponds to
having two Gibbs measures. Off this line, the average density is unique.
This corresponds to uniqueness of the Gibbs measure, in the Ising model;
see Theorem 10.2.
It is noteworthy that Exercise 8.17 shows that at critical pressure Pc
and critical temperature Tc free energy develops a singularity. However,
Theorem 4.15 states that the average density is unique at these values. In
fact, we will see in the next chapter that the Gibbs measure is unique for
the Ising model at these critical values when d = 2 and is expected to be
unique also when d = 3. The two definitions of phase transition seem then
to coincide everywhere but at this critical point. This is easily fixed by
slightly changing Definition 8.18 and saying instead that there is no phase
transition at (β, α) if and only if the Gibbs measure is unique for values in
a whole neighborhood of (β, α).
Remark 8.19. With the above modification, uniqueness of the Gibbs distribution
does coincide with the other definition of phase transition in many
important models (like the Ising model). However, it is not always a good
definition for phase transition in a common physical way of thinking, and
will not coincide with the other definition in a general setting. For example,

96 8. Formalism for classical lattice systems
the plane rotator model, where Ising spins are replaced by vectors on the
unit circle, exhibits a Kosterlitz-Thouless type of phase transition in which
above a certain βc, correlations no longer decay exponentially but only like
inverse powers; see [21]. On the other hand, it is also known (at least partly
rigorously, but based on older physics arguments) that there is a unique
Gibbs state; see [7], [29], and [20].
8.5. Observables
Microscopically, one can only observe finitely many spins. Hence, a microscopic
observable is defined as a real-valued FΛ-measurable function on Ω,
for some finite Λ ⊂ Z d .
Macroscopic observations, on the other hand, should not depend on
any finite collection of spins. Consequently, a macroscopic observable must
be measurable with respect to the tail σ-algebra T = ∩FΛc, where the
intersection is over all finite Λ ⊂ Zd . This is nicely expressed in the next
lemma.
Lemma 8.20. Let Π = {πΛ : Λ ⊂ Z d finite} be an arbitrary specification
and let f ∈ bF . Then f ∈ bT if and only if πΛf = f for all finite Λ ⊂ Z d .
Proof. Fix Λ ⊂ Zd finite. If f ∈ bT , then f ∈ bFΛc and Exercise 8.9
implies that πΛf = f. Conversely, if f ∈ bF is such that πΛf = f, then
f ∈ bFΛc. If this is true for all Λ finite, then f ∈ bT .
There is clearly a separation of microscopic and macroscopic scales.
More precisely, if f is both a microscopic and a macroscopic observable,
then it is of course constant.
Some people require that a macroscopic observable be shift invariant;
i.e. that it be I-measurable (see (7.1) for the definition of I).
Example 8.21. If f is a microscopic observable then the average
1
lim f ◦ θi
n→∞ |Vn|
i∈Vn
(whenever it exists) is both T - and I-measurable.
For a measure µ to qualify as a macrostate, macroscopic observables
should be deterministic under µ, because only microscopic quantities are
allowed to fluctuate. In other words, µ should satisfy a 0-1 law on the
relevant σ-algebra.
If we require macroscopic observables to be tail-measurable, then T
should be trivial under µ, but if we also require the observables to be shift
invariant, then µ should just be ergodic. (Exercise A.18 is relevant.)

8.5. Observables 97
In either case, the set of observable macrostates coincides with the set
of extreme Gibbs measures. (Recall the definition of an extreme point from
page 72.) Thus, extreme Gibbs measures are the physically reasonable objects,
and their study is central to statistical mechanics.
Theorem 8.22. Let Π be an arbitrary specification corresponding to an
absolutely summable shift-invariant interaction potential Φ.
(a) µ ∈ G Π is T -trivial if, and only if, µ is extreme in G Π .
(b) µ ∈ G Π ∩ Mθ(Ω) is I-trivial (and hence ergodic) if, and only if, µ
is extreme in G Π ∩ Mθ(Ω).
Proof. To prove (a) first consider µ ∈ G Π such that for some T -measurable
A we have 0 < µ(A) < 1. Define ν1(B) = µ(B|A) and ν2(B) = µ(B|Ac ).
Then ν1 = ν2 and µ = tν1 + (1 − t)ν2, where t = µ(A) ∈ (0, 1). Moreover,
ν1 and ν2 belong to G Π . Indeed, if Λ ⊂ Zd is finite then
ν1πΛ(B) = πΛ(σ, B)ν1(dσ) = µ(A) −1
1IA(σ)πΛ1IB(σ)µ(dσ)
= µ(A) −1
πΛ(σ, A ∩ B)µ(dσ) = µ(B|A) = ν1(B).
Above, we used Exercise 8.9 to write that 1IAπΛ1IB = πΛ1IA∩B. A similar
computation holds for ν2. Thus, µ is not extreme.
For the other direction suppose T is trivial under µ but µ = tν1 + (1 −
t)ν2, for ν1 and ν2 in G Π and t ∈ (0, 1]. Since ν1 ≪ µ, T is also trivial under
ν1. Then, by the backwards-martingale convergence theorem (see Theorem
6.1 on page 265 of [15]) we have that for any local f, πVnf converges, as
Vn grows to Z d , to E µ [f], µ-a.s., and to E ν1 [f], ν1-a.s. But again ν1 ≪ µ
implies that the limits agree, so µ = ν1 and µ is extreme.
Now we prove part (b). Let µ ∈ G Π ∩ Mθ(Ω) be such that for some
I-measurable set A we have 0 < µ(A) < 1. By Exercise A.18 there exists
a T -measurable set A such that µ(A∆ A) = 0. Thus, defining ν1 and ν2 as
above we have µ = tν1 + (1 − t)ν2 with ν1, ν2 ∈ G Π ∩ Mθ(Ω) and t ∈ (0, 1).
In other words, µ is not extreme. The other direction is a consequence of
the ergodic theorem and is left as an exercise; see Exercise A.16.
Gibbs measures can be written as a convex mixture of observable ones.
One way to see this is via Choquet’s theorem (see [31]). We give an alternate
approach.
Exercise 8.23. (Decomposition for Gibbs measures) Use limits of πΛn as
Λn ↗ Z d to construct a kernel with values in extreme Gibbs measures
and that gives versions of µ( · | T ) simultaneously for all µ ∈ G Π . Apply

98 8. Formalism for classical lattice systems
Exercise A.25 to prove that every µ ∈ G Π has a unique probability measure
Qµ supported on the extremes of G Π such that µ = ν Qµ(dν).
The set of stationary Gibbs measures can be empty even when the set of
Gibbs measures is not; see (11.46) in [22]. Sufficient conditions for existence
of stationary Gibbs measures are given in Section 5.2 of [22].
Exercise 8.24. (Decomposition for stationary Gibbs measures) Let µ ∈
G Π ∩ Mθ(Ω). Let κ be the kernel from Exercise A.26. Show that for any
Λ ⊂ Z d finite, κ(ω)πΛ = κ(ω) µ-a.s. Use Exercise A.25 to conclude the
existence of a unique probability measure Qµ supported on the extremes of
G Π ∩ Mθ(Ω) such that µ = ν Qµ(dν).
The above expresses the heuristic idea that even though macrostates are
the only states a system can be at, an experimenter will not know which
one it is and thus will consider that the system is in a weighted average of
macrostates.

Large deviations and
equilibrium statistical
mechanics
9.1. Replacing Hamiltonians with averages
Chapter 9
Large deviation theory is concerned with fluctuations of empirical averages
of microscopic quantities. Hence, it is intuitively reasonable that there is a
connection with equilibrium statistical mechanics. This connection can be
made precise as follows.
Let Φ be an absolutely summable shift-invariant continuous interaction
potential. The inverse temperature β will be fixed throughout this chapter
and thus we will set β = 1. To see β explicitly, one can simply replace each
Φ by βΦ. Define the microscopic observable
fΦ =
A:0∈A
ΦA
∈ Cb(Ω).
|A|
This is the energy contribution of the origin. fΦ ◦ θi is then seen as the
energy contribution of site i.
Now, the energy contribution of a volume is the sum of the energy contributions
of each of its sites. Of course, one can have different boundary
conditions, but boundary contribution should be of the same size as the
boundary itself. When the lattice is Z d , the boundary of Λ has size o(|Λ|).
This is summarized in Lemma 9.1.
99

100 9. Large deviations and equilibrium statistical mechanics
For ∆ ⊂ Λ ⊂ Zd finite, define Λ(∆) = {i ∈ Λ : i + ∆ ⊂ Λ} and
b∆,Λ = |Λ|
ΦA + Φ |Λ Λ(∆)|.
A:0∈A⊂∆
Note that for any finite ∆ ⊂ Z d ,
lim
Λ↗Zd b∆,Λ
|Λ|
=
A:0∈A⊂∆
ΦA ,
which can be made arbitrarily small by taking ∆ large.
Lemma 9.1 below reveals how large deviation theory sneaks in. It shows
how one can replace the Hamiltonian, in the definition of Gibbs measures,
by a volume average of shifts of fΦ.
Lemma 9.1. For ∆ ⊂ Λ ⊂ Zd finite,
sup H free
Λ (σΛ) −
(9.1)
fΦ(θiσ) ≤ b∆,Λ
and
(9.2)
Proof. Write
But
σ∈Ω
sup
H free
Λ =
i∈Λ
H
σ∈Ω
free
Λ (σΛ) − H σΛc A⊂Λ
=
ΦA =
ΦA
|A|
i∈Λ A:i∈A
A:i∈A
Thus,
H free
Λ (σΛ) −
fΦ ◦ θiσ
i∈Λ
ΦA
|A|
A⊂Λ i∈A
−
Λ (σΛ)
≤ b∆,Λ.
=
i∈Λ A:i∈A⊂Λ
ΦA
|A|
i∈Λ A:i∈A⊂Λ
ΦA
|A| .
ΦA
Φi+B
=
|A| |B|
B:0∈B
= fΦ ◦ θi.
≤
i∈Λ(∆) A:i∈A⊂Λ
≤
i∈Λ(∆) B:0∈B
i+B⊂Λ
≤
i∈Λ(∆) B:0∈B⊂∆
≤ b∆,Λ,
ΦA
|A|
ΦB
|B|
ΦB
|B|
+
i∈ΛΛ(∆) A:i∈A⊂Λ
+
i∈ΛΛ(∆) A:i∈A
+
i∈ΛΛ(∆) B:0∈B
ΦA
|A|
ΦA
|A|
ΦB
|B|

9.2. Thermodynamic limit of the pressure 101
and (9.1) is proved. Next write
H free
Λ (σΛ) − H σΛc Λ (σΛ)
≤
A∩Λ=∅
A∩Λ c =∅
≤
ΦA
i∈Λ(∆) A:i∈A⊂Λ
≤ |Λ(∆)|
≤ b∆,Λ,
B:0∈B⊂∆
ΦA +
i∈ΛΛ(∆) A:i∈A
ΦA
ΦB + |Λ Λ(∆)|
B:0∈B
ΦB
proving (9.2).
9.2. Thermodynamic limit of the pressure
Recall that λ is a fixed i.i.d. product measure on Ω. We defined the volumes
Vn = {i ∈ Zd : −n < i1, . . . , id < n} and we let τ ∈ Ω ∪ {free} be a given
boundary condition. The partition function was defined by
Z τ
Λ = e −Hτ Λ (σΛ)
λ(dσΛ).
A quantity that encodes all the thermodynamic properties of the system is
the pressure, defined as the limit (if it exists)
1
P (Φ) = lim
n→∞ |Vn| log Zτ Vn
of the finite volume pressure P τ 1
Λ (Φ) = |Λ| log Zτ Λ .
The term “pressure” makes sense in the grand-canonical ensemble of a
lattice gas (X = {0, 1} and φA(σ) = −J(A)
i∈A σi for some shift-invariant
positive function J). But, for example, for ferromagnetic models it is simply
the same as “minus the free energy per site.”
By (9.1), the fact that the limit P (Φ) exists follows from the large deviation
principle for empirical fields (Theorem 7.13) and Varadhan’s theorem
(page 32). The fact that this limit is independent of the boundary condition
τ is simply a result of (9.2). As a consequence, one also has a variational
principle defining the pressure P (Φ) in terms of specific relative entropy.
Recall the definition of p(f) in (5.1) (and Proposition 7.14).
Theorem 9.2. There exists an infinite volume pressure P (Φ) such that
(a) lim
sup
n→∞
τ∈Ω∪{free}
|P τ Vn − P (Φ)| = 0;
1
(b) P (Φ) = p(−fΦ) = lim
n→∞ |Vn| log Eλ P
−
[e i∈Vn fΦ◦θi ];

102 9. Large deviations and equilibrium statistical mechanics
(c) −P (Φ) = inf
ν∈Mθ(Ω) {Eν [fΦ] + h(ν | λ)}.
Proof. Theorem 7.13 gives us a large deviation principle for the laws of
the empirical fields under the i.i.d. measure λ. Then, taking E = M1(Ω),
X = M(Ω), and Y = Cb(Ω), Theorem 5.25 implies the existence of the limit
defining p(−fΦ) in part (b), as well as the variational formula in part (c),
provided we use (b) as the definition of P (Φ). Part (a) simply follows from
(9.1) and (9.2). Indeed, take ∆ = Vm and Λ = Vn, for n > m. Then
sup sup − H τ Vn (σVn) +
fΦ ◦ θiσ
≤ 2bVm,Vn.
In general
τ∈Ω∪{free} σ∈Ω
i∈Vn
log E λ [e f ] = log E λ [e g e f−g ] ≤ log E λ [e g ]e f−g = log E λ [e g ] + f − g .
Thus,
sup
τ∈Ω∪{free}
log Z τ Vn − log Eλ [e
− P
i∈Vn fΦ◦θi
] ≤ 2bVm,Vn.
Now divide by |Vn| and take n → ∞ then m → ∞.
9.3. Specific relative entropy
Define the finite-volume Gibbs measures with free boundary conditions by
π free
Λ (dσΛ) = e−Hfree Λ (σΛ)
Zfree Λ
Theorem 9.3. For ν ∈ Mθ(Ω), the limit
exists and is given by
(9.3)
λΛ(dσΛ).
1
h(ν | Φ) = lim
n→∞ |Vn| H(νVn | π free
Vn )
h(ν | Φ) = E ν [fΦ] + h(ν | λ) + P (Φ).
Furthermore, for any Gibbs measure γ ∈ G Φ ,
exists and h(ν | γ) = h(ν | Φ).
h(ν | γ) = lim
n→∞ H(νVn | γVn)
Remark 9.4. Theorem 9.2 says that the free energy −P (Φ) is the smallest
value of the average energy contribution of 0 less the physical entropy, where
the minimization takes place over all shift-invariant probability measures.
Theorem 9.3 then says that this minimum is attained at any shift-invariant
Gibbs measure. This is analogous to (8.5) in the Curie-Weiss model and to
part (a) of Exercise 6.21.

9.3. Specific relative entropy 103
∗ Exercise 9.5. Use (9.3) to prove that h(· | Φ) is affine, lower semicontinu-
ous, and has compact sublevel sets.
Proof of Theorem 9.3. Recall that when the index is the volume Vn, we
abbreviate things by simply using index n. Let g ∈ bFn.
E νn [g] − log π free
n (e g ) = E νn [g] − log E λn [e g−Hfree n ] + log Z free
n
= E νn [g − H free
n ] − log E λn [e g−Hfree
n ] + E νn [H free
n ] + log Z free
n .
Since H free
n ∈ bFn, we conclude that
1
|Vn| H(νn | π free
n ) = 1
|Vn| H(νn | λn) + 1
|Vn| Eνn [H free
n ] + P free
Vn .
The first term on the right-hand-side converges to h(ν | λ) by the definition
of specific entropy. The third term converges to P (Φ), by Theorem 9.2. The
second term is equal to Eν [Hfree
n /|Vn|] which, by (9.1), has the same limit
as |Vn| −1
i∈Vn Eν [fΦ ◦ θi] = Eν [fΦ]. This gives the existence of h(ν | Φ) as
well as (9.3).
Next, observe that if γ ∈ G Φ , g ∈ bFn, and πn is the Gibbs specification
corresponding to Φ and volume Vn, then
log E γn [e g ] = log E γ [πn(e g )]
λn E [eg−H = log
τ n]
Eλn[e−Hτ n] γ(dτ)
= log
E λn [e g−H free
n e Hfree
n −Hτ n]
Eλn[e−Hfree n eHfree n −Hτ n] γ(dτ)
≤ log Eλn [e g−Hfree
n ]
Eλn[e−Hfree n ]
≤ log π free
n (e g ) + 2bVm,Vn,
+ 2 sup H
τ
free
n − H τ n∞
where we used (9.2) in the last line, with m < n. Consequently,
and
E νn [g] − log π free
n (e g ) ≤ E νn [g] − log E γn [e g ] + 2bVm,Vn
H(νn | π free
n ) ≤ H(νn | γn) + 2bVm,Vn.
The lower bound can be shown similarly and one has
|H(νn | γn) − H(νn | π free
n )| ≤ 2bVm,Vn.
Thus, h(ν | γ) exists and equals h(ν | Φ).

104 9. Large deviations and equilibrium statistical mechanics
9.4. Large deviations under Gibbs kernels
Using Varadhan’s theorem and (9.2), one can deduce a uniform large deviation
principle for the empirical fields under Gibbs kernels, and thus also
under Gibbs measures. Recall the periodized empirical fields Rn, defined in
(7.2).
Theorem 9.6. For any Borel-measurable set A ⊂ Mθ(Ω),
(9.4)
− inf h(ν | Φ) ≤ lim
ν∈A◦ inf
n→∞ τ∈Ω∪{free}
≤ lim
sup
n→∞
τ∈Ω∪{free}
≤ − sup
ν∈A◦ h(ν | Φ).
1
|Vn| log πτ Vn { Rn ∈ A}
1
|Vn| log πτ Vn { Rn ∈ A}
Also, if γ ∈ G Φ , then the distributions γ{Rn ∈ ·} satisfy a large deviation
principle with rate function h(· | γ).
Proof. Note that
γ{
Rn ∈ A} =
πVn(τ, { Rn ∈ A})γ(dτ).
So if we replace Rn by Rn, the last statement follows from (9.4) and h(· | γ) =
h(· | Φ). It then follows for Rn from Exercises 7.5 and 2.17 and the tightness
of the rate function h(· | γ). Let us thus prove (9.4).
Define the probability measures ρn on Mθ(Ω) by
ρn(A) = 1
cn
1I{ Rn ∈ A}e −|Vn|E e Rn[fΦ] dλ,
where cn = Eλ
e−|Vn|E e
Rn[fΦ] . Since Theorem 7.13 gives a large deviation
principle for the laws of Rn under the i.i.d. measure λ with rate function
h(· | λ), Exercise 4.9 implies that the large deviation principle holds for ρn
with normalization |Vn| and rate function
I(ν) = E ν [fΦ] + h(ν | λ) − inf
µ∈Mθ(Ω) {Eµ [fΦ] + h(µ | λ)}
= E ν [fΦ] + h(ν | λ) + P (Φ) = h(ν | Φ).

9.5. Dobrushin-Lanford-Ruelle (DLR) variational principle 105
Now, the same approximation step as before allows us to switch from ρn to
the Gibbs kernels. To see this, write for τ ∈ Ω ∪ {free},
log ρn(A) = log 1IA( Rn)e −|Vn|E e
Rn[fΦ]
dλ − log e −|Vn|E e Rn[fΦ]
dλ
≤ log 1IA( Rn)e −Hτ
ndλ − log e −Hτ ndλ + 2 H τ n − |Vn|E e Rn [fΦ]
≤ log π τ n{ Rn ∈ A} + 2 H τ n − |Vn|E Rn [fΦ] + 2|Vn| E e Rn [fΦ] − E Rn [fΦ] .
Abbreviate f (r)
Φ = 0∈A,|A|≤r ΦA/|A|. Then, using Exercise 7.5 one sees
that
lim E e Rn [fΦ] − E Rn [fΦ] ≤ 2
ΦA + lim E e Rn (r)
[f Φ ] − ERn [f (r)
Φ ] n→∞
Then, for m fixed,
lim
inf
n→∞ τ∈Ω∪{free}
0∈A,|A|>r
= 2
0∈A,|A|>r
n→∞
ΦA −→ 0.
r→∞
1
|Vn| log πτ n{ 1
Rn ∈ A} ≥ lim
n→∞ |Vn| log ρn(A)
bVm,Vn
− 4 lim
n→∞ |Vn|
≥ − inf h(ν | Φ) − 4 lim
ν∈A◦ n→∞
bVm,Vn
.
|Vn|
Taking m → ∞ kills the right-most term and proves the lower large deviation
bound. The upper bound is similar.
9.5. Dobrushin-Lanford-Ruelle (DLR) variational principle
We have so far seen that shift-invariant Gibbs measures are all minimizers
in the variational formula for the pressure P (Φ); see Remark 9.4. The
natural question is whether or not the minimizers are all shift-invariant
Gibbs measures. This would follow from Theorems 9.2 and 9.3 if one shows
that shift-invariant solutions to h(γ | Φ) = 0 are Gibbs measures.
Dobrushin-Lanford-Ruelle variational principle. Fix a shift-invariant
absolutely summable continuous interaction potential Φ. Let γ ∈ Mθ(Ω).
The following are equivalent.
(a) γ ∈ G Φ ;
(b) h(γ | Φ) = 0;
(c) E γ [fΦ] + h(γ | λ) = −P (Φ) = inf{E ν [fΦ] + h(ν | λ) : ν ∈ Mθ(Ω)}.
Remark 9.7. The above variational principle shows that if phase transition
happens, then the large deviation rate function under Gibbs measures has

106 9. Large deviations and equilibrium statistical mechanics
multiple zeroes. We will see in the next chapter that this happens in the
multidimensional Ising model at low temperature.
Proof of the DLR variational principle. (a) implies (b) due to Theorem
9.3. Also, (b) is equivalent to (c) due to (9.3). The delicate part is to
show that (b) implies (a). Suppose (a) does not hold. Then, there exists
some integer ℓ such that γ = γπVℓ . Thus, H(γ | γπVℓ ) ≥ 2η, for some η > 0.
By Exercise 7.11, there is some integer k such that H(γVk | (γπVℓ )Vk ) ≥ η.
For a probability measure µ ∈ M1(Ω) and two sets A, B ⊂ Zd , let µ A B =
µB(· | FA) be the conditional distribution of {σi : i ∈ B} given {σi : i ∈ A}.
Lemma 9.8. Let ∆ ⊂ Λ ⊂ Γ ⊂ Zd be finite. Then
| (γπ∆) Λ∆
Λ ) dγ − H(γ Λ∆
Λ | (π free
where
H(γ Λ∆
Λ
d∆,Λ =
A:A∩∆=∅
A⊂Λ
ΦA .
Γ ) Λ∆
Λ
) dγ
≤ 4d∆,Λ ,
Here d∆,Λ bounds the energy contribution from outside Λ to the volume
∆. For fixed ∆ it decays to 0 as Λ grows to Z d .
Proof of Lemma 9.8. Define temporarily the ∆-volume Hamiltonian of
interactions confined to Λ,
H∆,Λ =
A:A∩∆=∅
A⊂Λ
Throughout this proof write, for a configuration σ ∈ Ω and a set A ⊂ Z d ,
σA = (σi)i∈A. Define π∆,Λ(σΛ∆) ∈ M1(ΩΛ) by
E π∆,Λ(σΛ∆) [ϕ] =
for ϕ a bounded FΛ-measurable function.
ΦA.
ϕ(ω∆, σΛ∆)e −H∆,Λ(ω∆,σΛ∆) λ∆(dω∆)
e −H∆,Λ(ω∆,σΛ∆) λ∆(dω∆)
In what follows, equalities hold almost surely. Note, however, that all the
finite-volume measures involved in the computations are mutually absolutely
continuous relative to λ (restricted to the volume in question). Hence, the
statements hold almost surely relative to any of these measures. Throughout
the rest of the proof ϕ is an arbitrary bounded FΛ-measurable function.
.

9.5. Dobrushin-Lanford-Ruelle (DLR) variational principle 107
Compute
E πfree
Γ [ϕ | σΛ∆]
=
=
where
ΩΓΛ
ΩΓΛ
Ω∆ ϕ(ω∆, σΛ∆)e−Hfree Γ (ω∆,σΛ∆,τΓΛ) λ∆(dω∆) λΓΛ(τΓΛ)
ΩΓΛ
Ω∆ e−Hfree Γ (ω∆,σΛ∆,τΓΛ) λ∆(dω∆) λΓΛ(τΓΛ)
Ω∆ ϕ(ω∆, σΛ∆)e−H∆,Λ(ω∆,σΛ∆)−Σ ′ −Σ ′′
λ∆(dω∆) λΓΛ(τΓΛ)
ΩΓΛ
Ω∆ e−H∆,Λ(ω∆,σΛ∆)−Σ ′ −Σ ′′ λ∆(dω∆) λΓΛ(τΓΛ)
Σ ′ =
A:A∩∆=∅
A⊂Λ,A⊂Γ
Since sup |Σ ′ | ≤ d∆,Γ, one has
E πfree
Γ [ϕ | σΛ∆]
= e C1
= e C1
= e C1
ΩΓΛ
ΦA and Σ ′′ =
A:A∩∆=∅
A⊂Γ
ΦA.
Ω∆ ϕ(ω∆, σΛ∆)e−H∆,Λ(ω∆,σΛ∆)−Σ ′′
λ∆(dω∆) λΓΛ(dτΓΛ)
ΩΓΛ Ω∆ e−H∆,Λ(ω∆,σΛ∆)−Σ ′′ λ∆(dω∆) λΓΛ(dτΓΛ)
Ω∆ ϕ(ω∆, σΛ∆)e−H∆,Λ(ω∆,σΛ∆) λ∆(dω∆)
ΩΓΛ e−Σ′′ λΓΛ(dτΓΛ)
Ω∆ e−H∆,Λ(ω∆,σΛ∆) λ∆(dω∆)
Ω∆ ϕ(ω∆, σΛ∆)e −H∆,Λ(ω∆,σΛ∆) λ∆(dω∆)
Ω∆ e−H∆,Λ(ω∆,σΛ∆) λ∆(dω∆)
= e C1 E π∆,Λ(σΛ∆) [ϕ],
with C1 = C1(σΛ∆) and sup |C1| ≤ 2d∆,Λ.
ΩΓΛ e−Σ′′ λΓΛ(dτΓΛ)
Exercise 9.9. Show, similarly to the above computation, that
with sup |C2| ≤ 2d∆,Λ.
π∆ϕ(σΛ∆, τΛ c) = eC2(σΛ∆,τ Λ c ) E π∆,Λ(σΛ∆) [ϕ],
Next, we show that
E γπ∆
[ϕ | σΛ∆] =
π∆ϕ(σΛ∆, τΛc) γ(dτΛc | σΛ∆).
Indeed, if ψ is bounded and FΛ∆-measurable, then
E γπ∆
[ϕψ] = π∆(ϕψ) dγ = ψ π∆ϕ dγ =
=
ψ E γ [π∆ϕ | FΛ∆] dγ =
E γ [ψ π∆ϕ | FΛ∆] dγ
ψ E γ [π∆ϕ | FΛ∆] d(γπ∆).
In the last equality we have used the fact that on FΛ∆, γ and γπ∆ coincide.
,

108 9. Large deviations and equilibrium statistical mechanics
Combining all of the above, one has
E γπ∆
[ϕ | σΛ∆] = π∆ϕ(σΛ∆, τΛc) γ(dτΛc | σΛ∆)
=
e C2(σΛ∆,τΛc ) π∆,Λ(σΛ∆)
E [ϕ] γ(dτΛc | σΛ∆)
= e C′ 2 (σΛ∆) E π∆,Λ(σΛ∆) [ϕ]
= e −C1(σΛ∆)+C ′ 2 (σΛ∆) E π free
Γ [ϕ | σΛ∆],
with sup |C ′ 2 | ≤ 2d∆,Λ. This implies that for g ∈ bFΛ,
sup log E
Λ∆
(γπ∆) Λ [e g ] − log E (πfree
Γ )Λ∆
Λ [e g ]
≤ 4d∆,Λ,
which implies the claim of the lemma by the variational formula for relative
entropy (Theorem 6.5).
Returning to the proof of the theorem, take k large enough such that
dVℓ,Vk < η/8. Fix m ∈ N and pick n = n(m) such that Vn is a disjoint union
of md translates of Vk: Vn = ∪md s=1 (is + Vk). Abbreviate ∆j = ij + Vℓ and
Λj = ∪ j
s=1 (is +Vk), for j = 1, . . . , md . Abbreviate π = πfree Vn
. Recall Exercise
7.11 and the conditional entropy formula from Exercise 6.14. Now compute
for j ≥ 2,
H(γΛj
| πΛj ) − H(γΛj−1 | πΛj−1 )
≥ H(γΛj | πΛj ) − H(γΛj∆j | πΛj∆j )
= H(γ Λj∆j
| π Λj
Λj∆j
) dγΛj
Λj
≥
H(γ Λj∆j
Λj
= H(γΛj | (γπ∆j )Λj ) − 4dVℓ,Vk
| (γπ∆j )Λj∆j
) dγΛj − 4dVℓ,Vk
Λj
≥ H(γij+Vk | (γπij+Vℓ )ij+Vk ) − 4dVℓ,Vk
≥ η − 4η
= η/2.
8
On the fourth line, we used the fact that (γπ∆j )Λj∆j = γΛj∆j . The above
computation also holds for j = 1, with the first line being H(γΛ1 | πΛ1 ).
Adding these inequalities over j gives
This implies that
H(γVn | π free
Vn ) ≥ md η/2.
1
|Vn| H(γVn | π free
Vn
) ≥ η
2|Vk| ,
for n = n(m). Letting m grow to infinity we see that h(γ | Φ) ≥ η
2|Vk| > 0,
which contradicts (b).

9.5. Dobrushin-Lanford-Ruelle (DLR) variational principle 109
As a corollary of the above variational principle, one can now deduce
that G Φ is not empty, even when the spin space X is not compact.
Corollary 9.10. If Φ is an absolutely summable shift-invariant continuous
interaction potential, then G Φ ∩ Mθ(Ω) is nonempty, convex, and compact.
Proof. We already know that G Φ ∩ Mθ(Ω) is convex. We have shown that
G Φ ∩ Mθ(Ω) = {γ : h(γ | Φ) = 0} = ∩n≥1{γ : h(γ | Φ) ≤ 1/n}.
By Theorem 9.6, h(· | Φ) is a rate function. Its infimum must then be 0
and the sets in the above intersection are nonempty. By Exercise 9.5, they
are also compact. The intersection is thus nonempty and compact, by the
finite intersection property of nested compact sets; see Proposition 4.21 of
[18].
Using the notion of subdifferentials, one can give a nice geometric interpretation
of a Gibbs state. To this end, set B and Mθ(Ω) in duality by
〈Φ, ν〉 = −E ν [fΦ]. Then, by part (c) of Theorem 9.2, P (Φ) is a supremum
over affine continuous functions and is hence convex and lower semicontinuous
on B.
Corollary 9.11. Suppose γ ∈ Mθ(Ω). Then, γ ∈ G Φ if, and only if, γ is a
tangent to P at Φ; i.e. γ ∈ ∂P (Φ).
Exercise 9.12. Show that the duality above between B and Mθ(Ω) satisfies
Assumption 5.1 so that the induced weak topologies are Hausdorff.

Phase transition in the
Ising model
Chapter 10
We saw in Section 4.3 that the Curie-Weiss model of ferromagnetism captures
the phase transition phenomenon known as spontaneous magnetization.
A serious shortcoming of the Curie-Weiss model is that every pair of
spins interacts with the same strength, so there are no spatial effects. A
more realistic model of ferromagnetism is the Ising model where the spins
are placed on the sites of an integer lattice and only neighboring spins interact.
This is still a crude approximation because in reality spins are carried
by electrons. But the Ising model is still a worthwhile test case for ideas of
statistical mechanics and it has been a rich source of deep mathematics.
Fix a dimension d. Let the state space be given by X = {−1, +1}. This
represents our assumption that only up or down spins are possible. The Ising
model on Z d has a nearest neighbor potential described by three parameters:
the strength of the nearest neighbor coupling (J > 0), the strength of the
external magnetic field (h ∈ R), and the inverse temperature (β > 0). The
potential takes the form
⎧
⎪⎨ −Jσiσj, A = {i, j}, with i − j1 = 1,
ΦA(σ) = −hσi,
⎪⎩
0,
A = {i},
otherwise.
111

112 10. Phase transition in the Ising model
Here i1 is the ℓ1 norm and thus i − j1 = 1 means that i and j are
nearest neighbors. Consequently, the Hamiltonian is
H ω Λ(σΛ) = −J
σiσj − J
σiωj − h
i,j∈Λ
i−j 1 =1
i∈Λ, j∈Λ c
i−j 1 =1
Taking the reference measure to be λ = ( 1
2δ−1 + 1
2
specification Πβ,h,J = {πΛ : Λ ⊂ Zd finite} by
where
πΛ(ω, A) =
1
Z ω Λ,β,h,J
ΩΛ
Z ω Λ,β,h,J =
i∈Λ
σi .
⊗Zd δ1) , one can define the
1IA(σΛ, ωΛ c) e−βHω Λ (σΛ) λΛ(dσΛ),
ΩΛ
e −βHω Λ (σΛ) λΛ(dσΛ)
is the partition function. We will use the notation µ ω Λ,β,h,J = πΛ(ω, ·) and
will call ω the boundary condition.
The free Hamiltonian and the finite-volume Gibbs measures with free
boundary conditions are defined as above, but without the boundary terms
ωj. This is equivalent to setting ωj = 0 and thus we will use the notation
H0 Λ (σΛ) and µ 0 Λ,β,h,J . We will also use the notation µ+
Λ,β,h,J for the case of
+ boundary condition, i.e. ωj = 1 for all j, and similarly µ −
Λ,β,h,J for the
Gibbs measure with boundary condition ω ≡ −1.
The mapping T σ = −σ is called the global spin flip.
∗Exercise 10.1. Check the following symmetry in the absence of an external
magnetic field: µ +
Λ,β,0,J = µ−
Λ,β,0,J ◦ T .
Already a first quick glance shows this model is reasonable. Indeed, J >
0 corresponds to ferromagnetism. When h = 0, H0 Λ has two ground states:
all spins equal to +1 and all spins equal to −1. At positive temperature and
without external field, µ +
Λ,β,0,J prefers + spins while µ−
Λ,β,0,J prefers − spins.
Based on what happens in the Curie-Weiss model, we expect that at low
temperatures the infinite-volume limits of the two measures give distinct
Gibbs states and thereby phase transition occurs. This would happen if
boundary effects can reach the origin regardless of how large the volume Λ
is. On the other hand, at high temp boundary effects should be swamped
by overwhelming noise and a unique Gibbs state is expected.
The free Gibbs measure µ 0 Λ,β,h,J prefers + or − spins depending on the
sign of h. When both the external magnetic field and boundary conditions
are present, the two compete. However, since the magnetic field acts on the
whole volume one would expect it to dominate boundary effects. So there
should not be any phase transition when h = 0.
Here is the main theorem, validating the above ideas.

10.1. One-dimensional Ising model 113
Theorem 10.2. The weak limits
µ +
β,h,J
= lim µ+
Λ↗Zd Λ,β,h,J
and µ−
β,h,J
exist for all d ∈ N, h ∈ R, J > 0, and β > 0.
= lim µ−
Λ↗Zd Λ,β,h,J
(a) When d = 1 there is a unique Gibbs measure for all h ∈ R, J > 0,
and β > 0 and in particular µ +
β,h,J
= µ−
β,h,J .
(b) When d ≥ 2 there exists a finite, positive critical temperature βc =
βc(J, d) such that this holds: if h = 0 or β < βc, then µ +
β,h,J = µ−
β,h,J
and the Gibbs measure is unique, while if h = 0 and β > βc, then
and we have a phase transition.
µ +
β,h,J
= µ−
β,h,J
The rest of this chapter is devoted to the proof of the above theorem.
This is a hard task and will take up a few sections.
Remark 10.3. When d = 2 it is known that βc = 1
2J log(1 + √ 2). See page
100 of [22]. Also, when d = 2 µ +
β,h,J and µ−
β,h,J are the only extreme Gibbs
measures. See Aizenman [1; 2] and Higuchi [24]. It then follows that all
Gibbs measures are shift invariant in dimension two. However, when d = 3
and h = 0 there exist values of β for which there are infinitely many extreme
Gibbs measures; see Dobrushin [13] and van Beijeren [40]. It also follows
from Dobrushin’s work that for d = 3, h = 0, and β large, there exist Gibbs
measures that are not shift invariant.
Remark 10.4. It is known that when d ≥ 2 but d = 3, the Gibbs measure
is unique for β = βc and h = 0. It is also believed, but is an open question,
that the same holds for d = 3; see [? ].
Remark 10.5. It is worthwhile at this point to take a second look at page
93 where the familiar liquid-gas phase transition was discussed.
10.1. One-dimensional Ising model
In this section we prove part (a) of Theorem 10.2. First we observe that the
compactness of Ω implies that the set of Gibbs measures G is nonempty for
all values of the parameters β > 0, J > 0, and h ∈ R; see Theorem 8.15.
Next, fix γ ∈ G , m ∈ N, and b−m, . . . , bm ∈ {−1, +1}. Consider the cylinder
set A = {σ ∈ Ω : σi = bi for |i| ≤ m}. Then, dominated convergence (see
page 16 of [15] or page 46 of [26]) implies that
γ(A) = lim
n→∞ EγE
γ [1IA|F [−n,n] c] = E γ
lim
n→∞ Eγ (10.1)
[1IA|F [−n,n] c] .
Next we derive the almost sure limit of ˜γn = Eγ [1IA|F [−n,n] c]. The existence
of this limit follows also from the backwards-martingale convergence theorem
(Theorem 6.1 on page 265 of [15] or page 155 of [26]).

114 10. Phase transition in the Ising model
Without loss of generality, we can absorb β into J and h and hence set
β = 1. Fix τ ∈ Ω and n > m. Write
˜γn(τ) = E γ [1IA|F [−n,n] c](τ) = µ τ [−n,n],1,h,J {σi = bi, |i| ≤ m}
exp
=
σ−n,...,σ−m−1
σm+1,...,σn
− H τ [−n,n] (σ−n, . . . , σ−m−1, b−m, . . . , bm, σm+1, . . . , σn)
ω−n,...,ωn
exp − H τ [−n,n] (ω−n,
.
. . . , ωn)
Define, for x, y ∈ {−1, +1}, A(x, y) = eJxy+h(x+y)/2 . Then an exponential
term in the numerator above can be rewritten as
−m−2
m−1
A(τ−n−1, σ−n)
i=−n
× A(bm, σm+1)
A(σi, σi+1)
n−1
i=m+1
A(σ−m−1, b−m)
A(σi, σi+1)
i=−m
A(bi, bi+1)
A(σn, τn+1)e −h(τ−n−1+τn+1)/2 .
If we think of A as a 2-by-2 matrix (called the transfer matrix)
A(−1, −1)
A =
A(1, −1)
A(−1, 1) eJ−h =
A(−1, −1)
e−J
and define
then we have
a0 =
m−1
i=−m
A(bi, bi+1),
e −J e J+h
˜γn(τ) = An−m+1 (τ−n−1, b−m) a0 A n−m+1 (bm, τn+1)
A 2n+2 (τ−n−1, τn+1)
The matrix A is real and symmetric. It has two real eigenvalues, say λ and
µ, with orthonormal eigenvectors u and v. Then
A = u v
λ
0
0
µ
u v
Observe that the determinant of A equals e 2J − e −2J > 0. Furthermore, its
trace is e J−h + e J+h > 0 and thus the two eigenvalues are positive. They
cannot be equal since otherwise A would be a multiple of the identity matrix.
We can, therefore, assume λ > µ > 0. Since (A/λ) k = uu T + (µ/λ) k vv T , we
see that the largest eigenvalue and its eigenvector determine the asymptotics
T
.
.

10.2. Phase transition at low temperature 115
of A. Define U = uu T and write
˜γn(τ) = a0λ −2m (A/λ)n−m+1 (τ−n−1, b−m)(A/λ) n−m+1 (bm, τn+1)
(A/λ) 2n+2 (τ−n−1, τn+1)
= a0λ −2m (U(τ−n−1, b−m) + o(1))(U(bm, τn+1) + o(1))
U(τ−n−1, τn+1) + o(1)
= a0λ −2m u(τ−n−1)u(b−m)u(bm)u(τn+1) + o(1)
u(τ−n−1)u(τn+1) + o(1)
Direct inspection shows that if u has a zero entry, then it cannot be an
eigenvector of A. Thus, ˜γn(τ) converges to a0λ −2m u(b−m)u(bm), which is
independent of τ. Hence, (10.1) implies that γ(A) = a0λ −2m u(b−m)u(bm).
This characterizes γ ∈ G and part (a) of Theorem 10.2 is proved.
In fact, we have proved a little bit more than absence of phase transition
in the one-dimensional model.
Exercise 10.6. Define the vector ρ(x) = u(x) 2 and the matrix
P (x, y) =
A(x, y)u(y)
, x, y ∈ {−1, +1}.
λu(x)
Check that P is a transition matrix and that ρP = P ; i.e. ρ is an invariant
measure for P . Check also that γ is precisely the law of the Markov chain
with marginal ρ and transition matrix P . In other words, this Markov chain
is the unique Gibbs measure for the one-dimensional Ising model.
Remark 10.7. One can think of the Ising model as a Markov field; i.e. a
stochastic process (Xi) i∈Z d where the distribution of Xi, given (Xj)j=i is
determined by the values of Xj for |j − i| = 1. Specifications then play the
role the transition matrix plays for Markov chains. Then, when d = 1, the
situation is similar to the Markov chain case and the stochastic process is
completely determined by the specification. However, this is no longer true
for d ≥ 2.
10.2. Phase transition at low temperature
It is the absence of phase transition in the one-dimensional case that led to
the dismissal of the Ising model as a model of ferromagnetism until 1936
when Peierls asked whether it is possible at low temperature, starting with
+ spins everywhere, to fluctuate to a state where most of the spins are
−. For this to happen, Peierls argued, droplets of − spin must “freeze”
to make the − state. Since spins align inside the droplet the energy of
a droplet is proportional to its circumference. There are at most k3 k−1
droplets of circumference k surrounding the origin. This means that the
likelihood of trapping the origin in a droplet of length more than k is at
.

116 10. Phase transition in the Ising model
most C
m≥k m3m−1 e −cmβ . If β is large enough this decays exponentially
fast with k. Therefore, droplets of − spins surrounding the origin are small,
which means boundary does influence the origin. Peierls concluded that the
answer to his question is in the negative and a phase transition must occur in
the two-dimensional Ising model for β large enough. We make this rigorous
in the proof of the next theorem.
Theorem 10.8. Let d = 2 and set h = 0. Then for all large enough β
the following holds: every weak limit point of the sequence {µ +
Λ,β,0,J : Λ ⊂
Zd finite} is distinct from every limit point of {µ −
Λ,β,0,J : Λ ⊂ Zd finite}.
Consequently there exists more than one Gibbs measure and phase transition
happens at h = 0 and large β.
This theorem is a first step towards proving part (b) of Theorem 10.2.
The rigorous proof is due to Griffiths [23] and Dobrushin [11].
Proof. Let us temporarily drop β > 0, h = 0, and J > 0 from the notation
and write µ ±
Λ = µ±
Λ,β,0,J . We expect the + boundary condition to make it
more likely for the spin at 0 to be +. In other words, we would like to prove
the following.
Claim 10.9. For some δ > 0, µ +
Λ {σ0 = −1} ≤ 1
2 − δ for all finite Λ ⊂ Z2 .
Let us assume this claim for the moment and finish the proof of the
theorem. By the spin-flip symmetry (Exercise 10.1)
µ −
Λ {σ0 = −1} = 1 − µ −
Λ {σ0 = 1} = 1 − µ +
Λ {σ0 = −1} ≥ 1
2 + δ
for all finite Λ ⊂ Z2 . Let µ + be an arbitrary limit point of {µ +
Λ }. Pick a
sequence Λn ↗ Z2 such that µ +
Λn → µ+ weakly. Then since {σ0 = −1} is
both open and closed
µ + {σ0 = −1} = lim
n→∞ µ+
Λn {σ0 = −1} ≤ 1
2 − δ.
By the same reasoning an arbitrary limit point µ − of {µ −
Λ } satisfies µ− {σ0 =
−1} ≥ 1
2 + δ. This implies that µ− = µ+ and the theorem is proved. It
remains to prove Claim 10.9.
Proof of Claim 10.9. Given a configuration σΛ ∈ ΩΛ with + boundary
condition, separate each nearest-neighbor {+, −}-spin pair with a horizontal
or vertical unit line segment drawn halfway between the sites, as in Figure
10.1. (Precisely speaking, these line segments are edges between nearestneighbor
points of the lattice (1/2, 1/2) + Z 2 . But we do not need a formal
discussion.) Due to the + boundary condition these line segments must form
closed circuits (or loops, or contours). If σ0 = −1, there is a unique smallest
such circuit that surrounds the origin and forms the boundary of the minus
cluster of the origin.

10.2. Phase transition at low temperature 117
-
+
+ + + + + + + +
+
+ + ++++
+
+
+
+
+ + + + + + +
+ - - - + +
+ - - - - - -
- + - +
+ -
+ + - - + +
+ + - - - + +
- + + + + -- +
- - + - - +
Figure 10.1. Peierls’ contours. The ellipse marks the minus spin at
the origin and the thick line shows the contour surrounding it.
For an arbitrary closed circuit γ surrounding the origin, let ΩΛ,γ denote
the event that σ0 = −1 and γ is the boundary of the minus cluster of the
origin. Let Γk be the set of closed circuits of length k surrounding the origin.
Then
µ +
Λ {σ0 = −1} ≤
∞
k=1 γ∈Γk
µ +
Λ (ΩΛ,γ).
Lemma 10.10. Setting as above. We have |Γk| ≤ k3 k−1 for all k ≥ 1. If
γ ∈ Γk then µ +
Λ (ΩΛ,γ) ≤ e −2kβJ .
Proof. This first claim is clear. Indeed, since 0 is trapped inside γ and γ
is a closed loop of length k, the contour must contain one of the vertical
line segments [(i + 1/2, −1/2), (i + 1/2, 1/2)] for some 0 ≤ i < k. Start by
choosing one of these k edges. Proceeding from (say) the top endpoint of
this edge, build the contour by choosing the remaining edges in order. At
each of the remaining k − 1 steps there are at most 3 available edges.
Suppose now that under configuration σΛ, γ ∈ Γk is the contour surrounding
the − cluster at 0. Let us flip all the spins inside γ to get the new
configuration (see Figure 10.2)
(10.2)
¯σj =
−σj if j is inside γ,
σj
if j is outside γ.
This mapping aligns every nearest-neighbor pair of spins separated by
γ, but the alignment or disagreement of a spin pair entirely inside or outside
γ is not affected. Consequently
J ¯σi¯σj =
Jσiσj
if γ does not separate i, j,
Jσiσj + 2J if γ separates i, j.

118 10. Phase transition in the Ising model
-
+
+ + + + + + + +
+
+ + ++++
+
+
+
+
+ + + + + + +
+ + + - + +
+ + + + + + + +
+ - -
- -
+ + + + + +
+ + + + + + +
- + + + + -- +
- - + - - +
Figure 10.2. The configuration resulting from applying (10.2) to Figure
10.1.
The same formula applies also to pairs {i, j} where one site lies outside Λ
and so its spin is part of the + boundary condition. For the total change of
the Hamiltonian we get
H +
Λ (¯σΛ) = −J
¯σi¯σj − J
¯σi
i,j∈Λ
i−j 1 =1
= −J
i,j∈Λ
i−j 1 =1
= H +
Λ (σΛ) − 2kJ.
i∈Λ,j∈Λ c
i−j 1 =1
σiσj − J
i∈Λ,j∈Λ c
i−j 1 =1
σi − 2kJ
2kJ is then seen to be the energy cost of the circuit. We have
µ +
Λ (ΩΛ,γ) =
σΛ∈ΩΛ,γ e−βH+ Λ (σΛ)
σΛ∈ΩΛ e−βH+
= e−2kβJ
Λ (σΛ)
σΛ∈ΩΛ,γ e−βH+ Λ (¯σΛ)
σΛ∈ΩΛ e−βH+ Λ (σΛ)
For a given γ the mapping (10.2) is one-to-one. Hence the sum in the
numerator of the right-most fraction contains a subset of the terms in the
denominator. Thus, the ratio is less than one and the lemma is proved.
To complete the proof of Claim 10.9, and thus of the theorem, write
µ +
Λ {σ0
∞
= −1} ≤ k3 k−1 e −2kβJ ∞
≤ e −2(βJ−log 3)k ≤ 1
− δ,
2
k=1
for β large enough.
Note that this argument fails in one dimension because the cost to flip
the spins in any interval is e −4βJ , while there are n 2 possible intervals surrounding
0 in (−n, n).
k=1
.

10.3. Uniqueness of phase at high temperature 119
10.3. Uniqueness of phase at high temperature
When β is small the Gibbs measure should be “close” to λ and one should
not have phase transition.
Theorem 10.11. Fix d ∈ N, J > 0, and h = 0. Then, if β > 0 is small
enough, G is a singleton.
This is the second step in the proof of part (b) of Theorem 10.2. Technically,
this is shown via a fixed point theorem. We will need some preliminaries
before we present the proof. We will omit β, J > 0, and h = 0 from
the indices in what follows.
Fix µ ∈ G . Let us start by defining the so-called correlation function for
µ as
for Λ ⊂ Z d finite.
ρ(Λ) = µ{σi = 1 ∀i ∈ Λ}
∗ Exercise 10.12. Use Show that ρ determines µ.
Hint: For finite, disjoint A, B ⊂ Z d expand the product in
1I{σ = 1 on A and σ = −1 on B}
=
1I{σ = 1 on A} − 1I{σ = 1 on A ∪ {i}} .
i∈B
∗Exercise 10.13. Fix Λ ⊂ Zd finite and i ∈ Λ. Let Ni = {j ∈ Zd :
i − j1 = 1}. Let B ⊂ Ni Λ. Use inclusion-exclusion to prove that
µ σk = 1 ∀k ∈ (Λ {i}) ∪ B, σj = −1 ∀j ∈ Ni (B ∪ Λ)
=
ρ((Λ {i}) ∪ A)(−1) |B| (−1) |A| .
A:B⊂A⊂NiΛ
Next, for i ∈ Zd and a collection of its nearest neighbors B ⊂ Ni = {j ∈
Zd : j − i1 = 1}, let µ B i be the Gibbs measure on the volume {i} with
boundary condition τj = 1 if j ∈ B and τj = −1 otherwise. Then for Λ and
V , two finite subsets of Zd , and for i ∈ Λ, define the Kirkwood-Salzburg
kernel, on the space of finite subsets of Zd , as
Ki(Λ, V ) = (−1)
|A|
B⊂A
(−1) |B| µ (Λ{i})∪B
{σi = 1},
if V = (Λ {i}) ∪ A for some A ⊂ Ni Λ. Otherwise we set Ki(Λ, V ) = 0.
Then operators K are contractions.
i

120 10. Phase transition in the Ising model
Lemma 10.14. If β > 0 is small enough, then for any finite Λ ⊂ Zd and
any i ∈ Λ
|Ki(Λ, V )| ≤ 1
2 .
V ⊂Z d finite
Proof. Choose β > 0 small enough so that
e
sup
−βJk
e−βJk −
+ eβJk Now write
V ⊂Z d finite
k,ℓ∈{0,±2,...,±2d}
|Ki(Λ, V )| =
A⊂NiΛ
B⊂A
e −βJℓ
e −βJℓ + e βJℓ
1
< .
24d
(−1) |B| µ (Λ{i})∪B
{σi = 1} .
In the sum over B inside the absolute value, group terms with |B| even
with terms with |B| odd. There is a one-to-one correspondence between
such terms and by the choice of β, all the terms µ (Λ{i})∪B
{σi = 1} are
i
within 1
24d of each other. Hence, the sum over B inside the absolute value is
bounded by 2 |A|−1 · 2−4d . Since |A| ≤ 2d, this is bounded by 2−(2d+1) . But
since |Ni| ≤ 2d, there are at most 22d possible sets A, and hence the whole
thing is bounded by 1/2.
Next, one observes that Ki’s fix the function ρ.
Kirkwood-Salzburg equation. Let ρ be the correlation function corresponding
to a Gibbs measure µ. For any Λ ⊂ Zd finite and any i ∈ Λ,
ρ(Λ) =
ρ(V )Ki(Λ, V ).
V ⊂Z d finite
Proof. Compute
ρ(Λ) =
µ σk = 1 ∀k ∈ B ∪ Λ, σj = −1 ∀j ∈ Ni (Λ ∪ B)
=
B⊂NiΛ
=
B⊂NiΛ
=
=
A⊂NiΛ
B⊂NiΛ
V ⊂Z d finite
µ σ ≡ 1 on (Λ {i}) ∪ B, σ ≡ −1 on Ni (B ∪ Λ)
µ (Λ{i})∪B
i {σi = 1}
ρ((Λ {i}) ∪ A)(−1)
ρ(V )Ki(Λ, V ).
A:B⊂A⊂NiΛ
|A|
B⊂A
i
× µ (Λ{i})∪B
i {σi = 1}
ρ((Λ {i}) ∪ A)(−1) |B| (−1) |A|
(−1) |B| µ (Λ{i})∪B
{σi = 1}
In the third equality we used Exercise 10.13.
i

10.4. Case of no external field 121
As mentioned earlier, a fixed point theorem now finishes the proof of
Theorem 10.11.
Proof of Theorem 10.11. Let µ, ˜µ be in G and denote by ρ and ˜ρ the
corresponding correlation functions. Fix a finite Λ ⊂ Zd and i ∈ Λ. Then,
|ρ(Λ) − ˜ρ(Λ)| =
Ki(Λ, V )[ρ(V ) − ˜ρ(V )]
V ⊂Z d finite
≤ 1
sup
2 V ⊂Zd |ρ(V ) − ˜ρ(V )|.
finite
Taking sup over Λ implies that ρ ≡ ˜ρ which implies that µ = ˜µ.
Exercise 10.15. Check that there is nothing special about h = 0 and that
one can actually prove that there is a unique Gibbs measure for any fixed h
and small β > 0.
10.4. Case of no external field
To complete Theorems 10.8 and 10.11 into a proof of part (b) for the case
h = 0 we make the following two informal observations.
First, increasing the temperature increases the effect of noise. Hence, if
at a given temperature there was no phase transition, the same should hold
for higher values of the temperature. In other words, if there is no phase
transition at a value β = β0, then there should not be a phase transition at
any lower value β ≤ β0.
Second, increasing dimension increases the size of the boundary relative
to the volume making it easier to have distinct Gibbs states. Thus, one
might expect that if there is phase transition in dimension d, then there will
also be phase transition in higher dimensions.
Before making this precise we develop some preliminary definitions and
results.
Let us put a partial order on elements of Ω. We say that ω ≤ σ if ωi ≤ σi
for all i ∈ Z d . A function f : Ω → R is increasing if f(ω) ≤ f(σ) whenever
ω ≤ σ. We can then define a partial order on M1(Ω) by saying that µ ≤ ν
(ν stochastically dominates µ) if E µ [f] ≤ E ν [f] for all increasing functions
f ∈ Cb(Ω).
Exercise 10.16. Check that this relation does define a partial order on
elements of M1(Ω); i.e. prove that if µ ≤ ν and ν ≤ µ, then µ = ν.
Hint: Observe that 1I{σ : σi = 1 ∀i ∈ A} is increasing.
We will need the following interesting fact. We defer its proof to Appendix
A.3.

122 10. Phase transition in the Ising model
Strassen’s lemma. Recall that Ω = {−1, 1} Zd.
Let µ, ν ∈ M1(Ω) be such
that µ ≤ ν. Assume that µ{σi = 1} = ν{σi = 1} for all i ∈ Zd . Then µ = ν.
The next theorem expresses the intuition that flipping some − spins to
+ spins at the boundary is favorable for + spins inside the volume.
Theorem 10.17. Fix d ∈ N, β > 0, J > 0, and h ∈ R. Also fix a finite
Λ ⊂ Z d . If ω ≤ σ, then µ ω Λ,β,h,J ≤ µσ Λ,β,h,J .
Proof. The proof uses Holley’s inequality, the proof of which is given in
Appendix C.1. Define pointwise minima and maxima by (η ∧ ξ)i = ηi ∧ ξi
and (η ∨ ξ)i = ηi ∨ ξi.
Holley’s inequality. Let ΩΛ = {−1, 1} Λ for some finite Λ ⊂ Z d . Let µ
and ν be strictly positive probability measures on ΩΛ. If
for all η and ξ in ΩΛ, then µ ≤ ν.
µ(η ∧ ξ)ν(η ∨ ξ) ≥ µ(η)ν(ξ)
Let us drop the indexes β, h, and J. According to the above theorem,
it is enough to prove that
This translates into
µ ω Λ(η ∧ ξ)µ σ Λ(η ∨ ξ) ≥ µ ω Λ(η)µ σ Λ(ξ).
H ω Λ(η ∧ ξ) + H σ Λ(η ∨ ξ) ≤ H ω Λ(η) + H σ Λ(ξ).
It is enough to show that the above inequality holds term by term, in the
sums defining the Hamiltonians. That is
(ηi ∧ ξi)(ηj ∧ ξj) + (ηi ∨ ξi)(ηj ∨ ξj) ≥ ηiηj + ξiξj, ∀i, j ∈ Λ,
(ηi ∧ ξi)ωj + (ηi ∨ ξi)σj ≥ ηiωj + ξiσj, ∀i ∈ Λ, ∀j ∈ Λ,
h(ηi ∧ ξi + ηi ∨ ξi) ≤ h(ηi + ξi), ∀i ∈ Λ.
The third line is obviously true. In fact there is equality there. Next, observe
that if ηi = −1 or ξi = 1, then ηi ∧ ξi = ηi and ηi ∨ ξi = ξi. Thus, in this
case there is equality in the second line. On the other hand, when ηi = 1
and ξi = −1 the second line reads σj − ωj ≥ ωj − σj, which is true since
ω ≤ σ.
As to the first line, it is again an equality unless ηi = ξj = 1 and
ξi = ηj = −1 or ηj = ξi = 1 and ξj = ηi = −1, in which case it reads
2 ≥ −2.
As a consequence of the above one sees that − and + boundary conditions
are special.
Theorem 10.18. Fix β > 0, J > 0, h ∈ R, and d ∈ N.

10.4. Case of no external field 123
(a) µ −
Λ,β,h,J ≤ µω Λ,β,h,J ≤ µ+
Λ,β,h,J for all Λ ⊂ Zd .
(b) µ +
Λ,β,h,J
finite.
≤ µ+
∆,β,h,J
and µ−
Λ,β,h,J
≥ µ−
∆,β,h,J
(c) As Λ increases to Z d measures µ +
Λ,β,h,J
verge weakly to a limit µ +
β,h,J
(d) µ +
β,h,J
and µ−
β,h,J
(e) µ −
β,h,J ≤ µ ≤ µ+
β,h,J
for all ∆ ⊂ Λ ⊂ Zd
(respectively, µ−
Λ,β,h,J ) con-
(respectively, µ−
β,h,J ).
are shift-invariant Gibbs measures.
for all Gibbs measures µ.
Proof. Part (a) is an immediate consequence of the previous theorem. Part
(b) follows from part (a) and the definition of a specification. Indeed, let
f ∈ Cb(Ω) be increasing. Then, part (c) of Definition 8.7 implies
E µ+
Λ,β,h,J [f] = E µ+
Λ,β,h,J E µω ∆,β,h,J [f]
≤ E µ+
∆,β,h,J [f].
Part (c) follows from part (b). To see this observe first that for any
finite volume V ⊂ Zd the sequence µ +
Λ,β,h,J {σi = 1, ∀i ∈ V } is decreasing as
Λ increases. Hence, this sequence converges to some value ρ(V ). Inclusionexclusion
then shows that for any cylinder set A ∈ FV , µ +
Λ,β,h,J (A) converges
to some νV (A), determined by {ρ(∆) : ∆ ⊂ V }. Measures νV ∈ M1(ΩV )
are consistent and Kolmogorov’s extension theorem (page 474 of [15] or
page 60 of [26]) produces a unique probability measure µ +
β,h,J that has these
marginals. This is the weak limit of the measures µ +
Λ,β,h,J on the full space Ω
because weak convergence follows from convergence of marginals (Exercise
7.1). The same argument works for µ −
β,h,J .
These limits are Gibbs measures due to Theorem 8.15 (taking νn = δω≡1
then νn = δω≡−1). The shift invariance follows from applying Exercise 8.3 to
obtain the inequalities µ +
V n−|i|,β,h,J
≥ µ+
Vn,β,h,J ◦θi = µ +
Vn+i,β,h,J
≥ µ+
V n+|i|,β,h,J
and then taking a limit. Similar reasoning for µ −
β,h,J . Here, Vn = {j ∈ Z d :
|j| < n}.
To prove part (e) let µ be a Gibbs measure. Let f ∈ Cb(Ω) be a bounded
increasing function. Let Vn be an increasing sequence of cubes exhausting
Z d . Then, dominated convergence (see page 16 of [15] or page 46 of [26])
implies that
E µ [f] = lim
n→∞ EµE
µ [f | FV c n ]
= lim
n→∞ EµE
µω
Vn,β,h,J [f]
≤ lim
n→∞ EµE
µ+
Vn,β,h,J [f] = E µ+
β,h,J [f].
Similar argument for the other inequality.
Proof of part (b) of Theorem 10.2 when h = 0. Observe that due to
(e) in the above theorem, phase transition at (β, 0, J) is equivalent to µ −
β,0,J =

124 10. Phase transition in the Ising model
µ +
β,0,J which, by shift invariance and Strassen’s lemma, is equivalent to
µ −
β,0,J {σ0 = 1} = µ +
β,0,J {σ0 = 1}. By (e), Exercise 10.1, and (c) this in
turn is equivalent to having µ +
β,0,J {σ0 = 1} > 1/2.
We invoke another useful inequality. The proof can be found in Appendix
C.2.
Griffiths’ inequality. Fix a finite volume Λ ⊂ Zd . Let E = {{i, j} : |i−j| =
1, i ∈ Λ} be the set of nearest-neighbor edges of Λ, including edges from Λ
to its complement. Define the function F : [0, ∞) E → [−1, 1] by
σΛ
F (J) =
σ0
exp {i,j}∈E Ji,j
σiσj
exp {i,j}∈E Ji,j
,
σiσj
σΛ
where J = (Ji,j) {i,j}∈E, the sums run over σΛ ∈ {−1, 1} Λ , and for j ∈ Λ we
set σj = 1 (i.e. the + boundary condition). Then,
∂F
∂Ji,j
(J) ≥ 0 ∀{i, j} ∈ E and ∀J ∈ [0, ∞) E .
∗ Exercise 10.19. Fix J > 0 and h = 0. Use Griffiths’ inequality to show
that E µ+
Λ,β,0,J [σ0] increases with β. Also, show that if V (d)
n = {i ∈ Z d : |i| ≤
n}, then
σ0 dµ +
V (d)
n ,β,0,J ≤
σ0 dµ +
V (d+1)
n
,β,0,J .
Conclude that µ +
β,0,J {σ0 = 1} increases with β and with d.
We have shown, in Theorem 10.8, that for J > 0, d = 2, and β > 0
large, one has a phase transition and thus µ +
β,0,J {σ0 = 1} > 1/2. By the
above exercise, one has a phase transition in any d ≥ 2, provided β is large
enough.
Now, fix J > 0 and d ≥ 2. Let βc = sup{β : |G Πβ,0,J | = 1}. By the
previous argument, βc < ∞. By Theorem 10.11, βc > 0.
Clearly, there is phase transition at any β > βc. If, on the other hand,
there were a phase transition at β < βc, then µ +
β,0,J {σ0 = 1} > 1/2 and
Griffiths’ inequality implies then that µ +
β ′ ,0,J {σ0 = 1} > 1/2 for all β ′ ≥ β.
This would imply that βc ≤ β which contradicts the choice of β. Part (b)
of Theorem 10.2 is thus proved, in the case h = 0.
10.5. Case of nonzero external field
In this section we finish the proof of part (b) of Theorem 10.2.
Fix J > 0 and β > 0. We will sometimes drop the dependence on these
parameters from our notation in this section. Recall the infinite-volume
pressure P (Φ) = P (h) from Theorem 9.2. In this section we first show

10.5. Case of nonzero external field 125
that if P is differentiable at h, then there is no phase transition at (β, h, J).
Then, we will show that P is differentiable for all h = 0. Let us start with
an exercise and a lemma.
We remind the reader again that subscripts Vn are abbreviated as subscripts
n; e.g. H + n = H +
Vn .
∗Exercise 10.20. Recall the finite-volume pressure functions with + and
− boundary conditions P ± n (h) = |Vn| −1 log Eλ [e−βH± n ]. Prove that
∂P ±
n
1
= βEµ± n,h σi .
∂h |Vn|
Lemma 10.21. One has
∂P
lim
n→∞
± n
∂h
i∈Vn
= βEµ± h [σ0].
Proof. We will treat the + boundary condition, the other one being similar.
Fix an integer m and use the fact that |VnVn−m|
|Vn| → 0 as n → ∞ to write
∂P
lim
n→∞
+ n
∂h
1
|Vn|
σi ,
= lim
n→∞ βEµ+
1
n,h σi = lim
|Vn|
n→∞
i∈Vn
βEµ+ n,h
i∈Vn−m
with the same series of equalities for the liminf. Observe next that if i ∈
Vn−m, then i + Vm ⊂ Vn. Then, by (b) and (d) of Theorem 10.18,
This implies
E µ+
h [σ0] = E µ+
h [σi] ≤ E µ+
n,h[σi] ≤ E µ+
i+Vm,h[σi] = E µ+
m,h[σ0].
E µ+
∂P
h [σ0] ≤ lim
n→∞
+ n
∂h
∂P
≤ lim
n→∞
+ n
∂h
≤ Eµ+ m,h[σ0].
The claim follows from the fact that E µ±
m,h[σ0] → E µ±
h [σ0] as m → ∞.
Now we will need the following result from real analysis, the proof of
which we leave to the reader.
∗ Exercise 10.22. Let fn be a sequence of differentiable convex functions
on an interval (a, b). Assume fn(y) → f(y) for all y ∈ (a, b) and that f ′ (x)
exists for some x ∈ (a, b). Prove that f ′ n(x) → f(x).
Hint: Use convexity to show that for y < x < z, we have
fn(x) − fn(y)
x − y
≤ f ′ n(x) ≤ fn(z) − fn(x)
.
z − x
Theorem 10.23. Consider the Ising model in dimension d ≥ 1 and fix
β > 0 and J > 0. If the infinite-volume pressure P (β, h, J) is differentiable
with respect to h at h = h0, then |G Πβ,h 0 ,J | = 1.

126 10. Phase transition in the Ising model
The proof requires the convexity of P + n in h. This is the familiar consequence
of Hölder’s inequality. We leave it as an exercise.
∗ Exercise 10.24. Prove that for each n, P + n (h) is convex in h.
Proof of Theorem 10.23. Fix J > 0 and β > 0. By Exercise 10.20
and Theorem 9.2 P + n is a sequence of convex differentiable functions that
converge, pointwise in h, to P (β, ·, J). By Exercise 10.22, if ∂
∂h P (h0) ex-
ists, then ∂
∂h Pn(h0) → ∂
∂h P (h0). But then Lemma 10.21 implies that
P (h0) = βE µ+
h 0 [σ0]. The same argument implies that P (h0) = βE µ−
h 0 [σ0].
Thus, µ +
h0 {σ0 = 1} = µ −
h0 {σ0 = 1} and Strassen’s lemma along with shift
= µ− . By part (e) of Theorem 10.18 this implies
h0
the uniqueness of the Gibbs measure.
invariance imply that µ +
h0
Since a convex function on R is differentiable at all but countably many
points, we know phase transition can occur at at most countably many values
of h for each fixed β > 0 and J > 0. To conclude the proof of Theorem 10.2
we prove that P (h) is differentiable at all h = 0. This is the claim of the
next theorem.
Theorem 10.25. Consider the Ising model in dimension d ≥ 1. Then, for
all β > 0 and J > 0, P (β, h, J) is differentiable in h at h = 0.
Proof. Fix β > 0 and J > 0. We start with the case h > 0. Define
Mn(h) = E µ+
1
n,h σi .
|Vn|
Since |Mn(h)| ≤ 1, there exists, for each fixed h, a subsequence nk such
that Mnk (h) → M(h). By the diagonal trick (see page 76), one can find one
subsequence that works simultaneously for all rational h ≥ 0. Denote this
subsequence by nk. Next, we will need the following inequality. We defer
the proof to Appendix C.3.
i∈Vn
Griffiths-Hurst-Sherman inequality. Consider the Ising model in d ≥ 1.
Fix β > 0, J > 0, and the volume Vn. Then, ∂2 Mn
∂h 2 ≤ 0 for h > 0.
The above inequality says that Mn is concave when h > 0. We will now
use a geometric argument to show that Mnk actually converges for all h > 0.
be two rational numbers such
Let h > 0 be irrational and let hi and h ′ i
that (i − 1)h/i < hi < h < h + i(h − hi) < h ′ i . Let ti ∈ (0, 1) be such that
h = tihi + (1 − ti)h ′ i . Using the fact that h′ i > h + i(h − hi) one finds that
ti > i/(i + 1). Hence, ti → 1 as i → ∞.
Similarly, let h ′′
i be a rational number in (0, hi − (i − 1)(h − hi)). Such
a number exists because hi > (i − 1)h/i. Choose t ′ i ∈ (0, 1) to have hi =
t ′ i h + (1 − t′ i )h′′
i . Then, t′ i > (i − 1)/i and t′ i
→ 1.

10.5. Case of nonzero external field 127
Now use the concavity of Mn to write
which implies
and thus
Similarly,
implies
Mnk (h) ≥ tiMnk (hi) + (1 − ti)Mnk (h′ i)
lim Mnk
k→∞
(h) ≥ tiM(hi) + (1 − ti)M(h ′ i)
lim Mnk (h) ≥ lim
k→∞
i→∞ M(hi).
Mnk (hi) ≥ t ′ iMnk (h) + (1 − t′ i)Mnk (h′′
i )
lim M(hi) ≥ lim Mnk (h).
i→∞
k→∞
Consequently, Mnk (h) does converge to some value M(h). Since the functions
Mnk are all concave for h > 0, their pointwise limit M must be concave
as well. But then, it is continuous for h > 0.
Exercise 10.20 implies that
P + n (h) − P + h
n (0) = β
0
Mn(s) ds.
Taking n to infinity along the subsequence nk and using dominated convergence
one concludes that
P (h) − P (0) = β
h
0
M(s) ds.
Since M is continuous on R {0}, P must be differentiable at h > 0.
When h < 0 one can use symmetry to write
E µ−
1
n,h σi = −E
|Vn|
µ+
1
n,−h
|Vn|
i∈Vn
Then, Exercise 10.20 implies that
P − n (0) − P − 0
n (h) = −β
h
i∈Vn
σi
Mn(−s) ds.
= −Mn(−h).
Once again, taking n to infinity shows that P is differentiable at h < 0.
The above theorem combined with Theorem 10.23 shows that there is
no phase transition when h = 0. This completes the proof of Theorem 10.2.

Percolation approach
to phase transition
Chapter 11
This chapter will introduce the Fortuin-Kasteleyn random cluster model and
use it to give alternative proofs of some of the Ising model results.
129

Part III
Further large
deviations topics

Further asymptotics
for i.i.d. random
variables
12.1. Refinement of Cramér’s theorem
Chapter 12
In this section X, X1, X2, . . . are i.i.d. real valued random variables with
nondegenerate law µ. Furthermore, Sn = X1 + · · · + Xn, for n ≥ 1, M(θ) =
E[e θX ], dom(M) = {θ : M(θ) < ∞}, and I(x) = sup θ∈R{θx−log M(θ)}, for
x ∈ R. We also write θ + = sup{θ : M(θ) < ∞} and θ − = inf{θ : M(θ) <
∞}.
Let θ− < 0 < θ + . Then m = x µ(dx) exists as a finite number, and for
all a > m
1
lim
n→∞ n log P {Sn/n ≥ a} ≤ −I(a),
and
1
lim
n→∞ n log P {Sn/n > a} ≥ −I(a).
If a < m, then
and
1
lim
n→∞ n log P {Sn/n ≤ a} ≤ −I(a),
1
lim
n→∞ n log P {Sn/n < a} ≥ −I(a).
This is a consequence of Cramér’s theorem; see page 23. However, here
we want to examine these probabilities at the nonlogarithmic scale. The
133

134 12. Further asymptotics for i.i.d. random variables
refinement of Cramér’s theorem is the following, where we now have nonlogarithmic
expressions for the large deviation probabilities P {Sn/n > a}
for a > m, and P {Sn/n < a} for a < m.
Theorem 12.1. If θ − < 0 < θ + and m < a < c + = lim θ↗θ + M ′ (θ)
M(θ)
(12.1)
0 < inf
n≥1
, then
√ nI(a) √ nI(a)
n e P {Sn/n > a} ≤ sup n e P {Sn/n > a} < ∞.
n≥1
Similarly, if θ − < 0 < θ + and c − = lim θ↘θ − M ′ (θ)
M(θ)
0 < inf
n≥1
< a < m, then
√ nI(a) √ nI(a)
n e P {Sn/n < a} ≤ sup n e P {Sn/n < a} < ∞.
n≥1
Remark 12.2. An upper bound on √ n e nI(a) P {Sn/n > a} is given in the
proof. See (12.6) below.
Proof. We will verify (12.1). The other statement then follows from replacing
X with −X. Hence let m < a < c + . Then
(12.2)
where
(12.3)
P {Sn/n > a} = e −n(θa−log M(θ)) Jn(θ),
Jn(θ) = · · ·
e −θ Pn j=1 (xj−a)
n
1I (xj − a) > 0
j=1
× eθ(x1+···+xn)
M(θ) n
µ(dx1) · · · µ(dxn).
Since µ is nondegenerate, direct differentiation shows M ′ (θ)
M(θ) is strictly increasing
and continuous on (0, θ + ). Thus the limit defining c + exists, and
since M ′ (0)
M(0) = m we indeed have m < c+ and m < a < c + implies there
exists a unique θa ∈ (0, θ + ) such that M ′ (θa)
M(θa) = a. Then, Exercises 3.6 and
3.7 give
and
(12.4)
Now
(12.5)
where
I(a) = sup{θa
− log M(θ)} = θaa − log M(θa),
θ≥0
P {Sn/n > a} = e −nI(a) Jn(θa).
Jn(θa) = E[e −θa e Sn 1I{ Sn > 0}],
Sn =
n
(Zj − a),
j=1

12.1. Refinement of Cramér’s theorem 135
and Z, Z1, Z2, . . . are i.i.d. with law νθa
dνθa eθax
(x) =
dµ M(θa) .
Letting Fn(u) = P { Sn ≤ u √ n}, we see
Jn(θa) =
Define
ρ =
=
=
having Radon-Nikodym derivative
e
(0,∞)
−√n θau
dFn(u)
∞ √
n θae
(0,∞) u
−√n θax
dx dFn(u)
∞
(Fn(x) − Fn(0))
0
√ n θae −√n θax
dx.
3 eθax
|x − a|
M(θa) µ(dx) and σ2
=
2 eθax
(x − a)
M(θa) µ(dx).
Now the Berry-Esseen theorem (see page 126 of [15]) implies
sup |Φ(x) − Fn(x)| ≤ 3
x
E[|Z − a|3 ]
σ3√ = 3ρ/(σ
n
3√ n),
where Φ(x) is the c.d.f. of a mean zero Gaussian random variable G with
variance σ2 . Hence
Jn(θa) ≤
∞
0
(Φ(x) − Φ(0)) √ n θae −√ n θax dx + 6ρ
σ 3√ n
∞ √
n θae −√n θax
dx,
and reversing the previous argument with Φ replacing Fn we see
Jn(θa) ≤
which implies
=
e
(0,∞)
−√n θax 6ρ
dΦ(x) +
σ3√n ∞
e
0
−√n θax−x2 /(2σ2 ) dx
√
2πσ2 √
n Jn(θa) ≤ (2πσ 2 θ 2 1
−
a)
0
+ 6ρ
σ 3√ n ,
2 + 6ρ
.
σ3 (12.6)
Thus the upper bound in (12.1) holds and it remains to check the lower
bound. Now
Jn(θa) ≥ e −2Aθa P {n −1/2 Sn
∈ (n −1/2 A, 2n −1/2 A)}
≥ e −2Aθa
P {G ∈ (n −1/2 A, 2n −1/2 A)} − 6ρ/( √ nσ 3
)
∼ e −2Aθa (A/ √ 2πσ 2 n − 6ρ/( √ nσ 3 ))
as n → ∞. Thus taking A sufficiently large we see
√
lim n Jn(θa) > 0.
n→∞

136 12. Further asymptotics for i.i.d. random variables
(12.1) then follows from observing that for each n ≥ 1 the quantity in
question is positive. For otherwise, we would have P (X > a) n ≤ P (Sn >
na) = 0 and X ≤ a P-a.s. But then M ′ (θ) ≤ aM(θ), for all θ ∈ (θ − , θ + ),
and c+ ≤ a which contradicts a < c + . The theorem is proved.
Remark 12.3. A similar argument yields the same result for P {Sn/n ≥ a}
and P {Sn/n ≤ a}, except one needs to consider the possibility that Sn has
an atom at zero, but that requires only minor adjustments.
Remark 12.4. If the law of Z has a density, or its characteristic function
φZ(θ) satisfies
lim
|θ|→∞ |φZ(θ)| < 1,
then more precise error estimates are possible than those obtained from
the Berry-Esseen theorem. These are the so-called Edgeworth expansions,
and when applicable they would imply asymptotic results for these large
deviation probabilities rather than the upper and lower bounds produced
here.
Remark 12.5. Formulas (12.1) and (12.5) provide a representation formula
for the corresponding large deviation probabilities, and their analogues are
known for random variables with values in R d , and even in a separable
Banach space, when the intervals (a, ∞) and (∞, a) are replaced by open
convex subsets D. One of the first difficulties in establishing such formulas
is to find a suitable replacement for the endpoint of the interval, when
the set D is of higher dimension. The replacement point is known as the
dominating point for the set D, and their existence and uniqueness is now
reasonably well understood. They have found applications to the Gibbs
conditioning principle, large and moderate deviation probabilities, and also
to the Nummelin conditional weak law of large numbers in the vector space
setting. In each of these applications the starting point is the analogue of
the formulas (12.1) and (12.5) for the relevant vector space setting.
12.2. Moderate deviations
Consider X1, X2, . . . i.i.d. real valued random variables. For convenience,
let X be an independent copy of these variables. Recall that for n ≥ 1,
Sn = X1 + · · · + Xn.
As we have seen on page 27, order 1 deviations of Sn/n are handled by
Cramér’s theorem while order n −1/2 deviations are described by the central
limit theorem. In this section we are concerned with deviations of order
n −1/2+α , for α ∈ (0, 1/2).
Define t + = sup{t : E[e tX ] < ∞} and t − = inf{t : E[e tX ] < ∞}.

12.2. Moderate deviations 137
Theorem 12.6. Let t − < 0 < t + , E[X] = 0, and 0 < σ 2 = E[X 2 ] < ∞.
Also assume {bn : n ≥ 1} is a sequence of positive numbers such that
Then for all a > 0
(12.7)
1
lim bn/n 2 = ∞ and lim
n→∞ n→∞ bn/n = 0.
lim
n→∞
n
b 2 n
log P (Sn/bn ≥ a) = − a2
.
2σ2 Remark 12.7. In the proof of the lower bound part of (12.7) we only use
the fact that 0 < σ 2 < ∞. However, in the upper bound for (12.7) we use
the fact that the moment generating function is finite near zero.
Proof. The upper bound is basically Chebyshev’s inequality (page 15 of
[15]). That is, for a > 0 and all s ≥ 0
(12.8)
(12.9)
P {Sn/bn ≥ a} ≤ P {bnSn/n ≥ ab 2 n/n} ≤ E[e sbnSn/n ]e −sab2 n/n .
Next we prove that
lim
n→∞
n
b 2 n
log E[e sbnSn/n ] = s 2 σ 2 /2.
To prove (12.9) first observe that by Taylor’s formula for all t there exists
θ = θ(t) ∈ (0, 1) such that et = 1 + t + t2
2 eθt . Hence we have
E[e sbnX/n
] = E 1 + sbnX/n + 1
2 (sbnX/n) 2 e θsbnX/n
,
where |θ| ≤ 1. Since E[X] = 0 we thus have
E[e sbnX/n
1
] = 1 + E 2 (sbnX/n) 2 e θsbnX/n
,
and since E[(tX) 2 e t|X| ] < ∞ for all t < min{|t − |, t + } the dominated convergence
theorem (page 16 of [15] or page 46 of [26]) easily implies
Hence
lim
n→∞
n
b 2 n
lim
n→∞ E[(sX)2e θsbnX/n ] = s 2 E[X 2 ] = s 2 σ 2 .
log E[e sbnSn/n n
] = lim
n→∞
2
b2 log E[e
n
sbnX/n ]
n
= lim
n→∞
2
= lim
n→∞
b 2 n
n 2
b 2 n
= 1
2 s2 σ 2 .
log 1 + 1
2E[(sX)2e θsbnX/n ]b 2 n/n 2
,
1
2 E[(sX)2 e θsbnX/n ]b 2 n/n 2

138 12. Further asymptotics for i.i.d. random variables
Thus (12.9) holds and (12.8) implies
lim
n→∞
Setting s = a/σ 2 yields
(12.10)
n
b 2 n
lim
n→∞
log P {Sn/bn ≥ a} ≤ s2σ2 − sa.
2
n
b 2 n
log P {Sn/bn ≥ a} ≤ − a2
.
2σ2 Thus we have the upper bound result for (12.7), and it remains to prove the
analogous lower bound.
Recall that [x] denotes the greatest integer smaller or equal to x. Set
pn = [t2n/( b2n n )], qn = [n/pn], and rn = bn
for t > 0. Next, fix a > 0 and
tqn
ε > 0 and note that
(12.11)
P {Spn/rn ≥ t(a + ε)} qn ≤ P {Spnqn/rn ≥ tqn(a + ε)}
= P {Spnqn/bn ≥ a + ε}.
Since pnqn ≤ n and Sn = Spnqn + (Sn − Spnqn) we have
(12.12)
P {Sn/bn ≥ a} ≥ P {Spnqn/bn ≥ a + ε, |Sn − Spnqn| < εbn}.
But 1 − pnqn/n ≤ pn/n ≤ t2n/b2 n. Thus,
n
b2 log P {|Sn − Spnqn| ≥ ε
n
√ n} ≤ n
b2
log E[(Sn − Spnqn)
n
2 ]/(nε 2
)
= n
b2
log (n − pnqn)σ
n
2 /(nε 2 (12.13)
) → 0
as n → ∞. By the independence of Sn−Spnqn and Spnqn, (12.12) and (12.13)
combine to yield
n
n
lim log P {Sn/bn ≥ a} ≥ lim log P {Spnqn/bn ≥ a + ε}.
n→∞
n→∞
b 2 n
Thus (12.11) implies
n
lim log P {Sn/bn ≥ a} ≥ lim
n→∞
n→∞
But
and
b 2 n
n
b 2 n
rn = bn
tqn
b 2 n
n
qn log P {Spn/rn ≥ t(a + ε)}.
b 2 n
qn = n
b2 [n/pn] ∼
n
n2
b2 npn
= bn
t[ n
bnpn
∼
] nt pn
∼ t −2
bn t
∼
nt
2n2 b2 n
∼ p 1
2
n .
Hence the central limit theorem implies
n
lim log P {Sn/bn ≥ a} ≥ t
n→∞
−2 (12.14)
log P {Gσ ≥ t(a + ε)},
b 2 n

12.2. Moderate deviations 139
where Gσ is normal with mean zero and variance σ2 . Letting ε ↓ 0, [a+ε, ∞)
increases to (a, ∞) and hence (12.14) implies
n
lim
n→∞ b2 log P {Sn/bn ≥ a} ≥ t
n
−2 (12.15)
log P {Gσ > ta}.
Now for s > 0
√
s+1/s
2πσ2P {Gσ > s} ≥
and hence
s
u2
−
e 2σ2 du ≥ s −1 (s+1/s)2
−
e 2σ2 ,
lim t
t→∞
−2 log P {Gσ > ta} = − a2
,
2σ2 which when combined with (12.15) implies the lower bound
lim
n→∞
n
b 2 n
log P {Sn/bn ≥ a} ≥ − a2
.
2σ2 Thus combining this lower bound with (12.10) implies (12.7), and the
theorem is proved.
Remark 12.8. Replacing Xi by −Xi yields the same result for P {Sn/bn ≤
a} when a < 0.

Large deviations for
Markov chains
13.1. Restricting entropies on product spaces
Chapter 13
In this section we prove a general theorem about entropies on product spaces
with conditions on the marginals. This result will be helpful both for Markov
chains in this chapter and for nonstationary independent random variables
later in the next chapter.
Let X and Y be Polish spaces, κ ∈ M1(Y), and y ↦→ ρ y a measurable
map (in other words, a stochastic kernel) from Y into M1(X ). Define two
probability measures on X ×Y by r y = ρ y ⊗δy and Q(dx, dy) = ρ y (dx) κ(dy).
For f ∈ Cb(X × Y) define a functional Λ by
(13.1)
Λ(f) = log E
Y
ry
[e f
] κ(dy) =
log
Y
e
X
f(x,y) ρ y
(dx) κ(dy).
For ν ∈ M1(X × Y) and α ∈ M1(X ) define
(13.2) I(ν) = sup
f∈Cb(X ×Y)
{E ν [f] − Λ(f)}
and
(13.3) J(α) = sup
g∈Cb(X )
{E α [g] − Λ(g)}.
The defining formula (13.1) of Λ works just as well for functions that depend
only on the variable x, and so Λ can be regarded also as a function on Cb(X ).
This is the sense in which Λ appears in definition (13.3). Let νX and νY
denote the X - and Y-marginals of a measure ν ∈ M1(X × Y).
141

142 13. Large deviations for Markov chains
Theorem 13.1. We have these identities for ν ∈ M1(X × Y) and α ∈
M1(X ):
(13.4) I(ν) =
H(ν | Q)
∞
if νY = κ
if νY = κ,
and
(13.5) J(α) = inf{I(µ) : µ ∈ M1(X × Y) and µX = α }.
Proof. We prove first (13.4). By Jensen’s inequality,
(13.6) I(ν) ≥ sup
f∈Cb(X ×Y)
{E ν [f] − log E Q [e f ]} = H(ν | Q).
Write νy for the conditional probability of ν, given y ∈ Y. If νY = κ,
ν
I(ν) = sup E y
[f] − log E ry
[e f ] κ(dy)
≤
f∈Cb(X ×Y)
Y
H(ν
Y
y | r y ) κ(dy) = H(ν | Q).
The last equality used the conditional entropy formula (Exercise 6.14). Taking
f ∈ Cb(Y) in (13.2) shows that I(ν) = ∞ if νY = κ. We have proved
(13.4).
We prove (13.5) first for the case of compact X and Y. By dominated
convergence, Λ is a strongly continuous function on the Banach space
Cb(X × Y), in other words, continuous when Cb(X × Y) is equipped with
the supremum norm. By Hölder’s inequality it is a convex function. It
follows that Λ is a weakly lower semicontinuous function on Cb(X × Y).
Here is the argument for this step. By continuity and convexity, the set
U = {f : Λ(f) ≤ s} is a strongly closed, convex set. By a separation
theorem a function g ∈ U c can be strictly separated from U with a linear
functional (see the Hahn-Banach separation theorem on page 42 or Theorem
3.12 in [33]). Thus each g ∈ U c lies in the weak interior of U c which makes
U c weakly open, and thereby U is weakly closed. That sets of type U are
weakly closed is exactly the definition of weak lower semicontinuity.
As a convex, weakly lower semicontinuous function Λ is equal to its
convex biconjugate. Let g ∈ Cb(X ). Below we can think of g also as a
function on X × Y, by composing it with the projection (x, y) ↦→ x.
Λ(g) = Λ ∗∗ (g) = I ∗ (g)
= sup{E νX [g] − I(ν) : ν ∈ M1(X × Y)}
= sup E α [g] − inf{I(ν) : νX = α} : α ∈ M1(X ) .

13.1. Restricting entropies on product spaces 143
The third equality above hides a couple steps. The dual of Cb(X × Y)
is the space M(X × Y) of finite signed Borel measures on X × Y. (This is
one of the Riesz representation theorems, see Section 7.3 in [18].) Definition
(13.2) defines I as a convex and lower semicontinuous function on the space
M(X × Y). By the proof of Theorem 6.5 I(ν) = ∞ unless ν is a probability
measure. Thus the supremum in the definition of I ∗ can be restricted to
probability measures ν as done above.
The function α ↦→ inf{I(ν) : νX = α} is again convex and lower semicontinuous
in the Cb(X )-generated weak ∗ topology of M(X ). Since X and Y
are assumed compact I is automatically tight, and so lower semicontinuity
follows from part (b) of the contraction principle (page 29). Thus taking
convex conjugates once more gives
J(α) = Λ ∗ (α) = inf{I(ν) : νX = α}
and completes the proof of (13.5) for compact X and Y.
To prove (13.5) for Polish X and Y, we begin by observing that we can
assume that X and Y are dense Borel subsets of compact metric spaces ¯ X
and ¯ Y. Here is the argument. By separability X has a totally bounded
metric ¯ d, and then the completion ( ¯ X , ¯ d) of (X , ¯ d) is compact (details can
be found in Theorem 2.8.2 in [14] or Lemmas 6.1-6.3 in [30]). As a Polish
space X also has a complete metric, in other words, (X , ¯ d) is a topologically
complete metric space. By Theorem 2.5.4 in [14] a metric space is topologically
complete if and only if it is a countable intersection of open sets (i.e.
a Gδ set) in its completion.
Definition (13.1) of Λ works just as well for functions f on ¯ X × ¯ Y,
because we can think of ρy and κ as probability measures on ¯ X and ¯ Y that
happen to satisfy κ(Y) = 1 and ρy (X ) = 1 κ-almost surely. Let Ī and
¯J denote the functions defined by (13.2) and (13.3) on M1( ¯ X × ¯ Y) and
M1( ¯ X ), respectively, with the supremums now over Cb( ¯ X × ¯ Y) and Cb( ¯ X ).
A probability measure α on X can be thought of as a measure on ¯ X , and
hence the proof for compact spaces gives
(13.7)
¯ J(α) = inf{Ī(¯ν) : ¯ν ∈ M1( ¯ X × ¯ Y), ¯ν ¯ X = α}.
By Lemma B.1 the supremum in the definition (13.3) of J(α) can just
as well be taken over g ∈ Ub, d ¯(X ), where Ub, d ¯(X ) is the space of bounded
uniformly continuous functions on (X , ¯ d). Functions in Ub, d ¯(X ) and Cb( ¯ X )
correspond to each other bijectively via restriction and unique extension,
hence ¯ J(α) = J(α). Since X × Y is dense in ¯ X × ¯ Y this same argument
gives Ī(ν) = I(ν) for probability measures ν on X × Y. Since Q(X × Y) = 1
and H(¯ν | Q) is finite only if ¯ν ≪ Q, (13.4) shows that Ī(¯ν) = ∞ unless

144 13. Large deviations for Markov chains
¯ν(X × Y) = 1 too. These facts combine to imply that for α ∈ M1(X ),
(13.7) is the same as (13.5).
Now, we apply the above to a Markov kernel.
Let C +
b (S) be the space of functions f ∈ Cb(S) that are stricly positive
and bounded away from 0. Let p be a stochastic kernel from S into S, in
other words, a Markov chain transition probability kernel. Define
(13.8) J(α) = sup
f∈C +
b (S)
log f
pf dα.
If q(x) is a measurable M1(S)-valued function of x ∈ S and α ∈ M1(S),
then define the push forward probability measure on S as
αq(A) = q(x, A) α(dx).
Theorem 13.2. Assume the state space S Polish and p(x) a measurable
M1(S)-valued function of x ∈ S. Then
(13.9) J(α) = inf H(α × q|α × p).
q:αq=α
Proof. Apply Theorem 13.1 with these choices: X = Y = S, ρ y = p(y) for
y ∈ S, κ = α. Then (13.4) says
J(α) = inf
ν H(ν | Q)
with infimum over ν ∈ M1(S × S) with both marginals equal to α, and
Q = α × p. (Think of the space S × S as Y × X rather than X × Y.) If
νY = νX = α, the conditional probability of x under ν, given y, defines a
kernel q that fixes α. Conversely, any such kernel defines a ν.
13.2. Large deviations
MORE TO COME...

Convexity criterion for
large deviations
Chapter 14
Let X and L be two real vector spaces in duality, endowed with their weak
topologies as described in the beginning of Chapter 5. Assumption 5.1 is
in force so that spaces X and L are Hausdorff spaces. In this section we
prove an abstract large deviation theorem for a sequence of Borel probability
distributions {µn}n∈N on a convex, compact subset X0 of X . The concrete
example to keep in mind is the one where X = M(S) is the space of realvalued
Borel measures on a compact metric space S, L = C (S) = Cb(S)
and X0 = M1(S) is the space of Borel probability measures on S. When
we apply this result we will be able to remove the compactness assumption
with a compactification, with the aid of exponential tightness.
Let 0 < rn ↗ ∞ be a normalizing sequence.
(upper) pressure by
For ϕ ∈ L define the
(14.1) ¯p(ϕ) = lim
n→∞
1
log e rn〈x,ϕ〉 µn(dx).
rn
This function is finite because each ϕ is bounded on the compact space X0.
By Jensen’s inequality ¯p is convex. Let J : X → [0, ∞] denote the convex
conjugate of ¯p:
(14.2) J(x) = sup{〈x,
ϕ〉 − ¯p(ϕ)},
ϕ∈L
x ∈ X .
If the limit in (14.1) exists we drop the bar and write
(14.3) p(ϕ) = lim
n→∞
1
log e rn〈x,ϕ〉 µn(dx).
rn
X0
X0
145

146 14. Convexity criterion for large deviations
For x ∈ X define upper and lower local rate functions by
(14.4) κ(x) = − inf
x ∈ G ⊂ X , G open
and
(14.5) κ(x) = − inf
x ∈ G ⊂ X , G open
lim
n→∞
lim
n→∞
1
rn
1
rn
log µn(G)
log µn(G).
Since the measures µn are supported on X0, for x ∈ X X0 we have
κ(x) = κ(x) = ∞. The functions κ and κ are [0, ∞]-valued and lower
semicontinuous. By Exercise 3.5 and the compactness of the space X0, if
κ = κ = κ then the LDP holds with rate function κ. The main theorem of
this section asserts that this is the case provided the pressure (14.3) exists
and κ is convex.
Baxter-Jain theorem. [5] Let {µn} be a sequence of Borel probability measures
on the compact convex subset X0 of X . Assume that the limit p(ϕ) in
(14.3) exists for all ϕ ∈ L and that κ is convex. Then the LDP holds for
{µn} with rate function J = κ = κ and normalization {rn}.
We proceed in the proof without the two assumptions, existence of (14.3)
and convexity of κ, as far as possible. When these two assumptions are used,
they are stated explicitly in the hypotheses of the lemma.
Lemma 14.1. J ≡ ∞ on X X0. For all x ∈ X0
(14.6) J(x) = κ ∗∗ (x) ≤ κ(x) ≤ κ(x).
The set {κ = 0} is not empty. Consequently J is a lower semicontinuous,
proper convex function.
Proof. Hahn-Banach separation implies that J ≡ ∞ on X X0. This is
left as Exercise 14.7 at the end of the section.
In (14.6) the last inequality is evident from definitions (14.4)–(14.5).
The middle inequality is part (b) of Proposition 5.12.
We claim that
(14.7) ¯p(ϕ) ≥ 〈x, ϕ〉 − κ(x) ∀x ∈ X , ∀ϕ ∈ L.
Given x and ϕ, and c < 〈x, ϕ〉, let G = {y ∈ X : 〈y, ϕ〉 > c}. Then
e rn〈y,ϕ〉
µn(dy) ≥ e rn〈y,ϕ〉 µn(dy) ≥ e rnc µn(G)
from which
X0
¯p(ϕ) ≥ c + lim
n→∞
Letting c ↗ 〈x, ϕ〉 verifies (14.7).
G
1
rn
log µn(G) ≥ c − κ(x).

14. Convexity criterion for large deviations 147
Taking supremum over x on the right in (14.7) gives ¯p ≥ κ ∗ , and another
round of convex conjugation gives
(14.8) J(x) ≤ κ ∗∗ (x).
Next we claim that
(14.9) ¯p(ϕ) ≤ κ ∗ (ϕ) = sup {〈x, ϕ〉 − κ(x)}.
x∈X0
The equality above is simply the definition. The inequality completes the
proof of (14.6) because it implies J(x) ≥ κ ∗∗ (x). The argument for (14.9)
will be needed shortly again so we separate it into the next lemma.
The claim that {κ = 0} = ∅ is left as Exercise 14.8. Thus J is not
identically infinite. J is convex and lower semicontinuous by its definition
(14.2).
Lemma 14.2. Let g : X0 → [−∞, ∞) be upper semicontinuous and extend
the use of the notation ¯p(g) to such functions by defining
1
(14.10) ¯p(g) = lim log e
n→∞
rng(x) µn(dx).
Then
(14.11) ¯p(g) ≤ sup {g(x) − κ(x)}.
x∈X0
rn
Proof. Denote the right-hand side of (14.11) by A(g). We can assume
A(g) < ∞. Let a > A(g) and ε > 0 such that a − ε > A(g). (We have not
ruled out A(g) = −∞ but a is real.) We claim that each x ∈ X0 has an
open neighborhood Gx such that for large enough n
(14.12)
e rng dµn ≤ e rna .
Gx
This suffices for the proof because by compactness we can cover X0 with a
finite collection Gx1 , . . . , Gxm of these neighborhoods. Then for large enough
n,
e rng m
dµn ≤ e rng dµn ≤ me rna .
X0
This implies ¯p(ϕ) ≤ a, and letting a ↘ A(g) verifies (14.11).
i=1
Gx i
Now to show (14.12). If g(x) = −∞ take Gx = {g < a} which is open
by upper semicontinuity. If −∞ < g(x) < ∞ [note that g(x) = ∞ is not
allowed] then pick Gx so that g(y) ≤ g(x) + ε/4 for y ∈ Gx. Since
X0
−κ(x) ≤ A(g) − g(x) < a − ε − g(x),

148 14. Convexity criterion for large deviations
looking at the definition of κ(x) we can shrink Gx to ensure that
lim
n→∞
1
rn
Consequently, for large enough n
log µn(Gx) ≤ a − ε/2 − g(x).
µn(Gx) ≤ exp rn(a − ε/4 − g(x))
and (14.12) follows.
Combined with the observation that κ = κ implies the LDP, Lemma
14.1 tells us that J = κ implies the LDP with rate function J. Our goal will
now be to show that if the pressure exists and κ is convex, J = κ. The next
definition is the key.
Definition 14.3. Let f : X0 → (−∞, ∞] be a lower semicontinuous convex
function. A point z ∈ X0 is a regular point of f if, for any δ > 0 and any
open set G ∋ z, there exist ψ ∈ L and an open set U such that z ∈ U ⊂ G
and these inequalities hold:
(14.13) inf
x∈X0U
and
f(x) − 〈x, ψ〉 > f(z) − 〈z, ψ〉
(14.14) |〈x, ψ〉 − 〈z, ψ〉| < δ for x ∈ U.
Exercise 14.4. Let f be a finite convex function on (a, b). Show that
z ∈ (a, b) is regular if and only if z is not an interior point of an interval
(c, d) on which f is of the form f(x) = αx + β.
The strict inequality in (14.13) is important. We illustrate the usefulness
of this notion by showing that J(z) = κ(z) at regular points z of J. Now we
need to assume that the limit (14.3) defining the pressure actually exists.
Lemma 14.5. Assume that the limit in (14.3) exists for all ϕ ∈ L. Then
J(z) = κ(z) at each regular point z of J.
Proof. Fix a regular point z, pick δ > 0 and a neighborhood G of z, and let
ψ ∈ L and neighborhood U be given by Definition 14.3. Define the upper
semicontinuous g : X → [−∞, ∞) by
g(x) = 〈x, ψ〉 · 1IU c(x) − ∞ · 1IU(x).
By Exercise 14.9 at the end of this section,
(14.15)
p(ψ) ≤ lim
n→∞
1
log e rn〈x,ψ〉
µn(dx) ∨ ¯p(g).
rn
U

14. Convexity criterion for large deviations 149
From the definition (14.2),
(14.16) p(ψ) ≥ 〈z, ψ〉 − J(z).
On the other hand, combining Lemma 14.2, the definition of g, inequality
(14.6), and property (14.13) of regularity gives
¯p(g) ≤ sup
x∈U c
{〈x, ψ〉 − κ(x)} ≤ sup
x∈U c
{〈x, ψ〉 − J(x)}
< 〈z, ψ〉 − J(z).
Consequently the second member of the maximum in (14.15) is irrelevant.
Starting with (14.16) and using the second property (14.14) of regularity,
1
〈z, ψ〉 − J(z) ≤ p(ψ) = lim log e
n→∞ rn U
rn〈x,ψ〉 µn(dx)
1
≤ 〈z, ψ〉 + δ + lim log µn(U),
n→∞ rn
from which, since U ⊂ G,
1
lim log µn(G) ≥ −J(z) − δ.
n→∞
rn
Since G is an arbitrary neighborhood of z and δ > 0 is arbitrary, −κ(z) ≥
−J(z) follows.
We can now state the main technical result of this section which shows
that regular points are plentiful. Recall that a point z is an extreme point
of a convex set K if z ∈ K and it has this property: if z = sx + (1 − s)y for
some x, y ∈ K and 0 < s < 1, then x = y = z. In other words, z cannot
lie on a line segment inside K unless it is an endpoint. The set of extreme
points of K is denoted by ex(K).
Theorem 14.6. Let f : X0 → (−∞, ∞] be a lower semicontinuous convex
function. Let ϕ ∈ L, c ∈ R, and define the affine function g(x) = 〈x, ϕ〉 + c.
Assume that g ≤ f on X0, and that the convex set A = {x ∈ X0 : f(x) =
g(x)} is nonempty. Then every extreme point of A is a regular point of f.
A is a closed subset of the compact space X0, and hence it is a compact
convex set. By the Krein-Milman theorem [33, Theorem 3.23] a compact
convex set is the closed convex hull of its extreme points. Thus A has
extreme points, and consequently f has regular points. Before proving Theorem
14.6 let us see how it completes the LDP.
Proof of Baxter-Jain’s theorem. By its definition κ is lower semicontinuous,
and the assumption is that it is also convex. Hence κ = κ ∗∗ and by
Exercise 5.7 for z ∈ X ,
κ(z) = sup{〈z, ϕ〉 + c : ϕ ∈ L, c ∈ R, 〈x, ϕ〉 + c ≤ κ(x) ∀x ∈ X }.

150 14. Convexity criterion for large deviations
Suppose 〈x, ϕ〉 + c ≤ κ(x) for all x ∈ X . We claim that then also
(14.17) 〈x, ϕ〉 + c ≤ J(x) for all x ∈ X .
This implies that κ ≤ J, and thereby κ = J since the opposite inequality
was in Lemma 14.1. Then we have J = κ = κ and as already pointed out,
this is enough for the LDP with rate J.
Inequality (14.17) needs to be verified only on X0 because J ≡ ∞ on
X X0. Let c1 be the (finite) infimum of the lower semicontinuous function
J(x) − 〈x, ϕ〉 on the compact set X0. Since the sets {x : J(x) − 〈x, ϕ〉 ≤
c1 + m −1 } are a nested family of nonempty compact sets, their intersection
A = {x : J(x) − 〈x, ϕ〉 = c1} is not empty. Let g(x) = 〈x, ϕ〉 + c1. We are
exactly in the situation of Theorem 14.6 and so ex(A) consists of regular
points of J. By Lemma 14.5, J(z) = κ(z) for all z ∈ ex(A).
To summarize, we have now for z ∈ ex(A) that
〈z, ϕ〉 + c ≤ κ(z) = J(z) = 〈z, ϕ〉 + c1.
Consequently c ≤ c1, and by the definition c1 = infx(J(x) − 〈x, ϕ〉) (14.17)
follows.
As the last item of this section we prove Theorem 14.6.
Proof of Theorem 14.6. Let ϕ ∈ L, g = ϕ + c and A = {f = g} as
given in the statement of the theorem. Let z ∈ ex(A), and let δ > 0 and
an open neighborhood G of z be given. We need to produce ψ ∈ L and a
neighborhood U of z such that (14.13)–(14.14) are satisfied.
Pick an open neighborhood U ⊂ G of z such that
|g(x) − g(z)| < δ/2 for x ∈ U.
Let K be the closed convex hull of the compact set A U. As a subset of
the compact space X0, K is compact. By Milman’s theorem [33, Theorem
3.25], ex(K) ⊂ A U. Since A is compact and convex, K ⊂ A. It follows
that z /∈ K. (If z ∈ K then z must be extreme in K since it is extreme in
the larger set A. But z ∈ U and ex(K) ⊂ A U.) By the Hahn-Banach
separation theorem there exists λ ∈ L such that
sup〈x,
λ〉 < 〈z, λ〉.
x∈K
By continuity, we can find an open set V ⊃ K and η > 0 such that
(14.18) sup〈x,
λ〉 < 〈z, λ〉 − η.
x∈V
Since U ∪ V is an open set and contains A, by compactness there exists
ε > 0 such that f − g ≥ ε on X0 (U ∪ V ). Again by compactness we can

14. Convexity criterion for large deviations 151
fix a > 0 small enough so that
Now set
a |〈x − z, λ〉| ≤ 1
2 (ε ∧ δ) for all x ∈ X0.
ψ = ϕ + aλ.
We check that (14.13)–(14.14) are satisfied. For (14.14), recalling that
g = ϕ + c,
|〈x, ψ〉 − 〈z, ψ〉| ≤ |g(x) − g(z)| + a |〈x − z, λ〉| < δ.
Condition (14.13) is checked in two parts. For x ∈ X0(U ∪V ), recalling
that f(z) = g(z),
This rearranges to
f(x) − g(x) ≥ ε ≥ a〈x − z, λ〉 + ε/2 + f(z) − g(z).
For x ∈ V , by (14.18) and f ≥ g,
and thus
f(x) − 〈x, ψ〉 ≥ f(z) − 〈z, ψ〉 + ε/2.
f(x) − g(x) − a〈x, λ〉 ≥ f(z) − g(z) − a〈z, λ〉 + aη
f(x) − 〈x, ψ〉 ≥ f(z) − 〈z, ψ〉 + aη.
This completes the proof of Theorem 14.6.
∗ Exercise 14.7. Use Hahn-Banach separation to show that J ≡ ∞ on
X X0.
∗ Exercise 14.8. Show that κ is a rate function that satisfies the upper
large deviation bound and the set {κ = 0} is not empty.
∗Exercise 14.9. For an, bn ≥ 0,
1
lim log(an + bn) ≤ lim
n→∞
n→∞
rn
1
rn
log an
∨ lim
n→∞
1
rn
log bn .
Exercise 14.10. Reprove Sanov’s theorem with the Baxter-Jain Theorem.
Can you do the same for the process-level LDP for i.i.d. random fields?

Nonstationary
independent variables
Chapter 15
15.1. Generalization of relative entropy and Sanov’s theorem
In this chapter we generalize Sanov’s theorem to a sequence of independent
but not identically distributed random variables. As in Section 6.2 the state
space of the random variables {Xk} is a Polish space S. We assume the
random variables are defined as coordinate variables on the sequence space
Ω = S N . In this section we need to discuss even more frequently than before
probability measures on the space M1(S) of probability measures on S, so
let us abbreviate M1 = M1(S) and M1(M1) will denote M1(M1(S)). As
throughout, M1 is given the weak topology generated by Cb(S). Generic
elements of M1, that is, probability measures on S, will be denoted by α,
β and γ, while κ will be a probability measure on M1, and µ and ν are
probability measures on the product space S × M1.
Let λ = {λk : k ≥ 1} be a sequence of probability measures on S, that is,
an element of M N 1 . Let P λ = λk denote the product probability measure
on Ω with marginals (λk), uniquely defined by the requirement that
P λ {X1 ∈ B1, X2 ∈ B2, . . . , Xn ∈ Bn} =
n
λk(Bk)
for Borel subsets Bk ⊂ S. Our interest lies in the large deviations of the
empirical measure
Ln = 1
n
δXk
n
k=1
k=1
153

154 15. Nonstationary independent variables
under P λ .
We begin by developing the relevant generalization of relative entropy.
Assume given a probability measure Ψ on M1. Define a joint distribution
µ on S × M1 by
(15.1) µ(dx, dα) = α(dx) Ψ(dα).
The S-marginal of µ can be regarded as the mean of Ψ:
(15.2) µS(B) = α(B) Ψ(dα) for Borel subsets B ⊂ S.
M1
Recall Definition 6.2 of relative entropy H.
Definition 15.1. Given Ψ ∈ M1(M1) and µ defined by (15.1), define the
following entropy for γ ∈ M1:
(15.3) K(γ | Ψ) = inf
ν H(ν | µ)
where the infimum is over probability measures ν on S ×M1 with marginals
γ and Ψ.
Here is the duality and other basic properties of K.
Theorem 15.2. (a) For γ ∈ M1
(15.4) K(γ | Ψ) = sup E
f∈Cb(S)
γ
[f] −
M1
log E α [e f
] Ψ(dα) .
(b) K(γ | Ψ) ≥ H(γ | µS) for all γ ∈ M1. K(γ | Ψ) = H(γ | µS) for all
γ ∈ M1 if and only if Ψ is a pointmass.
(c) K is a convex, tight rate function on M1 and K(γ | Ψ) = 0 if and
only if γ = µS.
Proof. (a) Apply Theorem 13.1 with the choices X = S, Y = M1, and
ρ α = α for α ∈ M1.
(b) For the inequality apply Jensen’s inequality inside the supremum in
(15.4), as in (13.6). If Ψ is a pointmass then K(γ | Ψ) = H(γ | µS) follows
from (15.4).
Suppose now that K(γ | Ψ) = H(γ | µS) for all γ ∈ M1. Let f be
a bounded Borel function on S that satisfies e f dµS = 1. By Jensen’s
inequality
(15.5)
M1
log E α [e f
] Ψ(dα) ≤ log
M1
E α [e f ] Ψ(dα) = 0.
Define the probability measure γ by dγ = ef dµS. Then
E γ [f] = H(γ | µS) = K(γ | Ψ) ≥ E γ [f] −
M1
log E α [e f ] Ψ(dα)

15.2. Proof of the large deviation principle 155
from which
log E α [e f ] Ψ(dα) ≥ 0.
M1
Thus we have equality in the Jensen inequality in (15.5). Then by the strict
concavity of log, the random variable α ↦→ Eα [ef ] must be degenerate, in
other words Eα [ef ] = 1 = E µS [ef ] for Ψ-a.e. α. We can find a countable
family F of bounded functions such that {ef : f ∈ F} separates measures,
and so this is enough for concluding that Ψ-a.e. α equals µS.
(c) Convexity is evident from either (15.3) or (15.4). K(µS | Ψ) = 0
follows from observing that for γ = µS, the right-hand side of (15.4) is
≤ 0 for all f, by Jensen’s inequality. The rest follows from K(γ | Ψ) ≥
H(γ | µS).
Definition 15.3. A sequence λ = {λk : k ≥ 1} ∈ MN 1
the limit
is called regular if
(15.6)
n 1
Ψ(λ) = lim δλk
n→∞ n
exists in the weak topology of M1(M1).
The main theorem of this section asserts that regularity is necessary and
sufficient for the LDP.
Theorem 15.4. The distributions P λ {Ln ∈ · } satisfy an LDP with normalization
{n} if and only if λ is regular. In this case the rate function is
K( · | Ψ(λ)).
This LDP appeared in [4] and was generalized and further elucidated
in [35] and [36]. In particular, process level versions exist. Applications to
statistical mechanics, in the spirit of our Sections 4.3 and 6.3 and Chapter
9, appear in [37] and [38].
k=1
15.2. Proof of the large deviation principle
We begin with a small generalization of the LDP proved in Chapter 14. As
in that chapter, let X and L be two real vector spaces in duality, endowed
with their weak topologies, assumed Hausdorff, and let X0 be a convex,
compact subset of X . Let I be an arbitrary index set, and for each i ∈ I
let {µ (i)
n : n ∈ N} be a sequence of Borel probability measures on X0. Let
0 < rn ↗ ∞ be a normalizing sequence.
For x ∈ X define the upper and lower local rate functions by
(15.7) κ (i) (x) = − inf
x ∈ G ⊂ X , G open
lim
n→∞
1
rn
log µ (i)
n (G)

156 15. Nonstationary independent variables
and
(15.8) κ (i) (x) = − inf
x ∈ G ⊂ X , G open
lim
n→∞
1
rn
log µ (i)
n (G).
Assume that the sequences {µ (i)
n : n ∈ N} all determine the same pressure:
for all ϕ ∈ L and i ∈ I,
(15.9) p(ϕ) = lim
n→∞
1
rn
log
X0
e rn〈x,ϕ〉 µ (i)
n (dx).
Let J : X → [0, ∞] denote the convex conjugate of p:
(15.10) J(x) = sup{〈x,
ϕ〉 − p(ϕ)},
ϕ∈L
x ∈ X .
Define
(15.11) κ0(x) = sup κ
i∈I
(i) (x).
Theorem 15.5. Assume that the limit p(ϕ) in (15.9) exists for all ϕ ∈ L
and i ∈ I, and that κ0 is convex. Then for each i ∈ I the sequence {µ (i)
n }n∈N
satisfies the LDP with rate function J = κ (i) = κ (i) = κ0 and normalization
{rn}.
∗ Exercise 15.6. Go through Chapter 14 to check what more needs to be
said to prove Theorem 15.5. Not much, you should discover.
To start the proof of Theorem 15.4, assume first that limit (15.6) holds
and also temporarily that S is compact. We apply Theorem 15.5 to prove
the LDP. Take X to be the vector space of real-valued Borel measures on
S, X0 = M1 and L = Cb(S). The normalization sequence is naturally
rn = n. The index set is I = Z+ = {0, 1, 2, . . . }, and the measures µ (i)
n are
distributions of shifted empirical measures:
(15.12) µ (i)
n (B) = P λ
1
n
in+n
k=in+1
δXk
∈ B , B ⊂ M1 measurable.
We begin by verifying assumption (15.9). A linear functional ϕ ∈ L
is now identified with a function f ∈ Cb(S) and the duality is given by

15.2. Proof of the large deviation principle 157
integration: 〈α, ϕ〉 = f dα for α ∈ X . Consequently
1
n log
e n〈α,ϕ〉 µ (i)
n (dα) = 1
n log
in+n
exp
(15.13)
X0
= 1
n
in+n
k=in+1
= i + 1
(i + 1)n
−→
M1
log E λk [e f ]
in+n
k=1
as n → ∞, by assumption (15.6).
Ω
log E λk [e f ] − i
in
k=in+1
in
k=1
log E α [e f ] Ψ(λ, dα) ≡ p(f)
f(Xk) dP λ
log E λk [e f ]
Convexity of κ0 will follow from verifying that, for all i ∈ Z+ and probability
measures α, β and γ ∈ M1 such that γ = (α + β)/2,
(15.14) κ (i) (γ) ≤ 1
2 κ(2i) (α) + 1
2 κ(2i+1) (β).
To complete the argument for convexity of κ0 from (15.14), note that the
right-hand side is bounded above by κ0(α)+κ0(β) /2, then take supremum
over i on the left, and apply Exercise 5.9.
To prove (15.14), let c > −κ (i) (γ), and pick an open neighborhood G of
γ such that
in+n
1
∈ G .
1 λ
(15.15) c > lim log P
n→∞ n
We can assume that G is of the form
n
k=in+1
δXk
G = {ρ ∈ M1 : |E ρ [fℓ] − E γ [fℓ]| ≤ 4ε, ℓ = 1, . . . , L}
for some functions f1, . . . , fL ∈ Cb(S) and ε > 0. Define neighborhoods of α
and β by
and
G1 = {ρ ∈ M1 : |E ρ [fℓ] − E α [fℓ]| ≤ ε, ℓ = 1, . . . , L}
G2 = {ρ ∈ M1 : |E ρ [fℓ] − E β [fℓ]| ≤ ε, ℓ = 1, . . . , L}.
Let m = ⌊n/2⌋. Do a quick calculation to check that, if n is large enough,
(15.16)
in+n
1
fℓ(Xk) −
n
k=in+1
1
(2i+1)m
1
· fℓ(Xk)
2 m
k=2im+1
− 1
(2i+2)m
1
·
2 m
fℓ(Xk)
≤ ε
k=(2i+1)m+1

158 15. Nonstationary independent variables
uniformly for all values {Xk}. (Since fℓ is bounded, the issue is only which
terms appear in the sums.) This and independence of the {Xk} under P λ
imply that
P λ
1
n
in+n
k=in+1
≥ P λ
1
m
= P λ
1
m
δXk
∈ G
(2i+1)m
k=2im+1
(2i+1)m
k=2im+1
δXk ∈ G1 , 1
m
δXk
∈ G1
(2i+2)m
k=(2i+1)m+1
P λ
1
m
δXk
(2i+2)m
k=(2i+1)m+1
Apply n −1 log and let n → ∞ to continue from (15.15):
1 λ 1
c > lim log P
n→∞ n n
in+n
k=in+1
≥ 1
1 λ 1
· lim log P
2 m→∞ m m
δXk
(2i+1)m
k=2im+1
+ 1
1 λ 1
· lim log P
2 m→∞ m m
≥ − 1
2 κ(2i) (α) − 1
2 κ(2i+1) (β).
∈ G
δXk
(2i+2)m
∈ G1
k=(2i+1)m+1
Letting c ↘ −κ (i) (γ) completes the proof of (15.14).
δXk
∈ G2
δXk
∈ G2
∈ G2 .
We have verified the hypotheses of Theorem 15.5. Consequently we have
the LDP for the sequence {µ (0)
n } with the rate function J = p ∗ defined by
(15.10). Calculation (15.13) identifies the pressure p(f), and then a glance at
(15.4) confirms that J = K( · | Ψ(λ)) as claimed in Theorem 15.4. We have
completed the proof of the if-part of Theorem 15.4 under the assumption
that S is compact.
It remains to lift the compactness assumption. Let now S be Polish. We
check exponential tightness.
Lemma 15.7. Assume (15.6). Then for each b > 0 there exists a compact
subset Kb ⊂ M1 such that for all n ∈ N
(15.17) P λ {Ln ∈ K c b } ≤ e−bn .

15.2. Proof of the large deviation principle 159
∗Exercise 15.8. Show that Γ ↦→ ρΓ(·) =
α(·) Γ(dα) defines a contin-
M1
uous mapping from M1(M1) to M1. This follows fairly directly from the
definition of weak topology on probability measures.
Proof of Lemma 15.17. Pick a positive sequence εj ↘ 0 and define cj =
ε −1
j (j + 1 + log 2). By the above exercise and assumption (15.6) the probability
measures
¯λn = 1
n
λk
n
k=1
converge weakly. A convergent sequence is tight, so we may pick compact
sets Aj ⊂ S such that ¯ λn(Ac j ) ≤ e−cj for all n. By an exponential Chebyshev
inequality and the arithmetic-geometric inequality (a special case of Jensen’s
inequality) and the definition of cj,
n
P λ {Ln(A c j) ≥ εj} ≤ P λ
≤ e −nεjcj
n
k=1
= e −nεjcj
E ¯ λn e cj1I Ac j
k=1
1IA c j (Xk) ≥ nεj
E λk e cj1I Ac
j −nεjcj ≤ e
n
1
n
n
k=1
≤ e −nεjcj 2 n = e −n(j+1) .
E λk e cj1I Ac
j
n Let Kb = {α ∈ M1 : α(A c j ) ≤ εj for j ≥ b }. Kb is closed by the portmanteau
theorem (Exercise A.3) and compact because the defining condition
forces tightness. And finally,
P λ {Ln ∈ K c b
} ≤ P λ {Ln(A c j) > εj} ≤ e −bn .
j≥b
For the final step of the proof of the LDP we compactify, obtain the LDP
on the compactification, and then argue that the LDP can be restricted to
the original space.
Let ¯ S be the compactification of S obtained by completing S under a
totally bounded metric ¯ d, as was done in the proof of Theorem 13.1. S
is a dense Borel subset of ¯ S. M1 can be considered as a subset of M1( ¯ S)
because every measure on S is also a measure on ¯ S. As a subspace of M1( ¯ S)
the space M1 has its original topology. This is because the weak topology
of M1( ¯ S) is generated by Cb( ¯ S), which is in bijective correspondence with
U b, ¯ d (S) via restriction and unique extension of functions. And in fact we
can write ¯ M1 for M1( ¯ S) because it is the closure of M1 (Exercise 15.14).
From these considerations follows that the limit in assumption (15.6) is
also valid in the space M1( ¯ M1). The first part of the proof then gives us
an LDP on the space ¯ M1 with rate function K( · |Ψ(λ)). Let G be an open
subset of M1 and A a closed subset of M1. By the definition of relative

160 15. Nonstationary independent variables
topology, there exist an open set G1 ⊂ ¯ M1 and a closed set A1 ⊂ ¯ M1 such
that G = G1 ∪ M1 and A = A1 ∪ M1. Since Ln ∈ M1 P λ -a.s., the LDP on
the larger space ¯ M1 gives us the bounds
1
(15.18) lim
n→∞ n log P λ {Ln ∈ G} ≥ − inf K(α |Ψ(λ))
α∈G1
and
(15.19) lim
n→∞
1
n log P λ {Ln ∈ A} ≤ − inf K(α |Ψ(λ)).
α∈A1
The lower bound on the space M1 follows now because the right-hand side
of (15.18) cannot increase if G1 is replaced by G. To replace A1 with A on
the right-hand side of (15.19) we need to show that
(15.20) K(α |Ψ(λ)) = ∞ for α ∈ ¯ M1 \ M1.
This is a consequence of exponential tightness: (15.17) and the lower bound
give
inf
α∈Kc 1
K(α |Ψ(λ)) ≥ − lim
b
n→∞ n log P λ {Ln ∈ K c b } ≥ b.
Since Kb ⊂ M1 we can let b ↗ ∞ to get (15.20).
To summarize, (15.18)–(15.19) turn into the LDP on the space M1
claimed in Theorem 15.4, and the proof of the if-part is complete.
Next we prove the only if-part of Theorem 15.4, namely that existence of
an LDP implies the existence of the limit in (15.6). First a technical lemma.
Lemma 15.9. Let Z be a compact metric space, and Φ and Ψ Borel probability
measures on M1(Z ). Assume that
(15.21)
log E α [e g
] Φ(dα) = log E α [e g ] Ψ(dα)
M1(Z )
for all g ∈ Cb(Z ). Then Φ = Ψ.
M1(Z )
Remark 15.10. By a compactification argument this lemma can be proved
for Polish spaces but we need it only for compact spaces.
Proof. Let h ∈ Cb(Z ). Let δ > 0 be small enough so that δ h∞ < 1.
Then for u ∈ (−δ, δ) let g(x) = log(1 + uh(x)), and expand:
log E α [e g
] Φ(dα) = log(1 + uE α [h]) Φ(dα)
M1(Z )
= −
M1(Z )
∞ (−u) k
k=1
k
(E
M1(Z )
α [h]) k Φ(dα).

15.2. Proof of the large deviation principle 161
The last expression is an analytic function of u ∈ (−δ, δ), and consequently
the coefficients of its power series expansion are uniquely determined. We
can conclude that
(E α [h]) k Ψ(dα) ∀h ∈ Cb(Z ).
(E
M1(Z )
α [h]) k Φ(dα) =
M1(Z )
From this and the power series expansion for the exponential we deduce
the equality of these characteristic functions:
e iEα
[h]
Φ(dα) = e iEα [h]
Ψ(dα) ∀h ∈ Cb(Z )
M1(Z )
M1(Z )
where i = √ −1 is the imaginary unit. Take linear combinations h =
m
k=1 tkgk for g1, . . . , gm ∈ Cb(Z ) and vary the vectors (t1, . . . , tm) ∈ R m
to conclude that any vector of the form (E α [g1], . . . , E α [gm]) has the same
distribution under Φ(dα) and Ψ(dα).
Integrating term by term, it follows that
p(E
M1(Z )
α [g1], . . . , E α [gm]) Φ(dα) =
p(E
M1(Z )
α [g1], . . . , E α [gm]) Ψ(dα)
for any polynomial p in m variables and g1, . . . , gm ∈ Cb(Z ). Finally we use
compactness. By the Stone-Weierstrass theorem [18, Section 4.7] this class
of functions is dense among continuous functions on M1(Z ). Consequently
we have
F (α) Φ(dα) = F (α) Ψ(dα)
M1(Z )
M1(Z )
for all continuous F on M1(Z ) which implies the result.
Now assume that distributions P λ {Ln ∈ · } satisfy an LDP with normalization
{n} and some rate function I on M1. With λ fixed, our goal is
to prove that the probability measures
Ψn = 1
n
converge weakly on the space M1(M1).
To get limit points for {Ψn} for free we transfer the discussion to the
compact space M1( ¯ S). By the contraction principle the LDP holds for the
distributions P λ {Ln ∈ · } on the space M1( ¯ S) with rate function
(15.22) J(α) =
n
k=1
δλk
I(α), α ∈ M1
lim M1∋β→α I(β), α ∈ M1( ¯ S) \ M1.
(This situation was addressed in Exercise 4.3. See also Exercise 15.14 below.)
Since M1( ¯ S) is compact, so is M1(M1( ¯ S)). By standard metric space
arguments convergence of the sequence {Ψn} follows if we can show that

162 15. Nonstationary independent variables
there is a unique limit point. So let Ψ = limj→∞ Ψnj be the limit of some
subsequence {Ψnj }. Let g ∈ Cb( ¯ S). Integrating the bounded, continuous
function F (α) = log Eα [eg ] against the measure Ψnj and taking the weak
limit gives
1
nj
log E λk g
[e ]
(15.23)
M1( ¯ S)
log E α [e g ] Ψ(dα) = lim
j→∞
= lim
j→∞
nj
k=1
1
nj
log E λ
exp
nj
k=1
g(Xk) .
On the other hand, Varadhan’s theorem (page 32) together with the assumed
LDP gives
1
lim log Eλexp
n→∞ n
n
k=1
g(Xk) = sup
γ∈M1( ¯ {E
S)
γ [g] − J(γ)} = J ∗ (g).
Thus the quantities in (15.23) are uniquely defined and equal J ∗ (g), for all
limit points Ψ, and then by Lemma 15.9 there can be only one limit point.
We have proved that there is a limit Ψn → Ψ in the space M1(M1( ¯ S)).
It remains to argue that this limit is actually an element of the space
M1(M1), or in other words that Ψ{α : α(S) = 1} = 1. Let ¯α denote
the mean of Ψ:
¯α(B) = α(B) Ψ(dα) for B ⊂ S measurable.
M1( ¯ S)
It is enough to show that ¯α(S) = 1 because this implies
0 = ¯α( ¯
S \ S) = α( ¯ S \ S) Ψ(dα).
M1( ¯ S)
To show this last point, start with
E γ
(f) −
J(γ) ≥ J ∗∗ (γ) = sup
f∈Cb( ¯ S)
M1( ¯ S)
log E α [e f
] Ψ(dα) ≥ H(γ | ¯α)
where the last inequality came from part (b) of Theorem 15.2. Now rate
function I cannot be identically infinite on M1 because the upper bound
of the LDP implies inf I = 0. Then ¯α(S) > 0 because otherwise every γ
supported on S would fail γ ≪ ¯α and the inequality above would force
I(γ) = ∞. Combining Exercise 6.15 with the inequality above gives for
γ ∈ M1
I(γ) = J(γ) ≥ H(γ | ¯α) ≥ − log ¯α(S) ≥ 0.
Choose a sequence γj ∈ M1 such that I(γj) ↘ 0. This forces ¯α(S) = 1, and
completes the proof of Theorem 15.4.

15.2. Proof of the large deviation principle 163
Exercise 15.11. [36] Find the necessary and sufficient condition on λ under
which P λ {Ln ∈ · } satisfies the LDP with rate given by relative entropy
H( · | ρ) with respect to some probability measure ρ.
Exercise 15.12. [36] With notation as in (15.1)–(15.2), let p be the stochastic
kernel from S into M1(M1) defined by
1IA(x)p(x, B) µS(dx) = µ(A × B)
S
for Borel sets A ⊂ S and B ⊂ M1. Show that for α ∈ M1,
K(α | Ψ) = H(α | µS) + K(Ψ | α ◦ p −1 ).
Note that α ◦ p −1 is a probability measure on M1(M1) so the second Kentropy
makes sense. In particular, K(α | Ψ) = H(α | µS) if and only if
αp = Ψ.
∗ Exercise 15.13. Supply the missing details for the argument that Ψ-a.e.
α equals µS in the proof of part (b) of Theorem 15.2.
∗ Exercise 15.14. Show that probability measures on S are dense among
probability measures on ¯ S. (Hint: convex combinations of point masses are
dense among probability measures.) This provides justification for formula
(15.22).

Appendixes

Topics from probability
Appendix A
Ultimately, this will be an appendix on general probability: a 3-page summary
of graduate probability, probability space, measure, σ-algebra, main
limit theorems, conditional probabilities, monotone class, etc.
A.1. Weak convergence of probability measures
Let (X , d) be a metric space. Let M(X ) be the space of bounded signed
Borel measures and M1(X ) the subspace of probability measures. Let Cb(X )
be the space of bounded continuous functions on X .
Definition A.1. A sequence of probability measures µn ∈ M1(X ) converges
weakly to µ ∈ M1(X ) if f dµn → f dµ for all bounded continuous
functions f.
Since knowing f dµ for all bounded continuous functions uniquely determines
µ, µn cannot converge weakly to two different measures.
For f ∈ Cb(X ) and µ ∈ M(X ), let 〈µ, f〉 = f dµ. Then, M(X ) and
Cb(X ) are in duality and convergence of probability measures in topology
σ(M(X ), Cb(X )) corresponds to weak convergence, as defined above. By
Exercise 6.1 this weak topology is Hausdorff.
This notion of weak convergence has other useful characterizations.
Definition A.2. A family of bounded measurable functions {gi} determines
weak convergence on M1(X ) if weak convergence of µn to µ is equivalent to
convergence of gi dµn to gi dµ for all i.
∗ Exercise A.3. (Portmanteau theorem) Let µn be a sequence of probability
measures. Prove that the following are equivalent.
167

168 A. Topics from probability
(a) µn converge weakly to µ.
(b) f dµn → f dµ for all f ∈ U b, e d (X ), where U b, e d (X ) is the space
of bounded uniformly continuous functions on (X , d) and d is any
metric that defines on X the same topology as d.
(c) limn→∞ µn(F ) ≤ µ(F ) for closed sets F .
(d) lim n→∞ µn(G) ≥ µ(G) for open sets G.
(e) limn→∞ µn(A) = µ(A) for continuity sets of µ; i.e. measurable sets
A such that µ(A A ◦ ) = 0.
Hint: (a) ⇒ (b) is trivial. Use x ↦→ 1/(1 + d(x, F )) k to prove that (b) ⇒
(c). Then prove that (c) ⇔ (d) and that (c) and (d) ⇒ (e). For fixed ε > 0
choose {ai} N i=1 so that µ{f = ai} = 0, − sup |f| − 1 = a0 < a1 < · · · <
aN−1 < aN = sup |f| + 1, and ai − ai−1 < ε. Now prove that (e) ⇒ (a).
Unless X is finite, one cannot metrize M(X ).
Exercise A.4. Let U and V be two vector spaces in duality and let them
induce weak topologies on each other.
(a) Prove that if U is metrizable, V must have a countable algebraic
basis.
Hint: If U is metrizable, there is a countable collection Bk of
neighborhoods around 0, of the form in part (a) of Proposition 5.5,
such that every open neighborhood of 0 contains some Bk. Let
{vk} be the entire, at most countable, collection of vectors in V
appearing in the expressions of the Bk’s. Show that each v ∈ V is
a linear combination of finitely many vk’s. You may need to use
Lemma 3.9 of [33].
(b) Prove that M(R) and M(N) are not metrizable.
Now, if X is separable, then it is homeomorphic to a subset of a compact
metric space and thus there exists a metric d that defines the same topology
on X and under which X is totally bounded. Its completion under this
metric is then compact and U b, e d (X ) is separable under the sup norm; see
Lemmas 6.1-6.3 in [30] and Theorem Theorem 2.8.2 in [14]. Thus, there
exists a dense (in the sup norm) countable set {gn} ⊂ U b, e d (X ). By (b) in
the portmanteau theorem, weak convergence in M1(X ) is then metrized by
the metric
(A.1)
δ(µ, ν) =
1
2
n≥1
n gn∞
gn dµ −
gn dν
∗ Exercise A.5. Let X be a metric space. Let {gk} be any countable collection
of Cb(X )-functions that determines weak convergence on M1(X ).
.

A.1. Weak convergence of probability measures 169
Prove that on M1(X ) the weak topology generated by Cb(X ) coincides with
the topology given by the metric δ defined in (A.1).
Hint: Recall part (a) of Proposition 5.5 and use the equivalence between
(a) and (b) in the portmanteau theorem.
The above exercise shows that δ metrizes the weak topology on M1(X )
generated by Cb(X ). One, however, has other metrics that are defined without
reference to some unknown set {gn}. Define the Prohorov metric ρ on
M1(X ) by
(A.2)
ρ(µ, ν) = inf{ε > 0 : µ(F ) ≤ ν(F ε ) + ε for all closed F ⊂ X }.
Here, F ε = {x : d(x, y) < ε for some y ∈ F }.
Exercise A.6. Show that ρ is a metric.
Exercise A.7. Assuming that X is separable, show that ρ metrizes the
topology on M1(X ) induced by the weak topology σ(M(X ), Cb(X )).
Hint: Since the weak topology is metrizable, it is enough to prove that
ρ(µn, µ) → 0 is equivalent to µn → µ.
Exercise A.8. Let X = Z. Show that the Prohorov metric is given by
ρ(µ, ν) = 1
|µ(k) − ν(k)|.
2
k∈Z
Once one has a topology, one can ask when a set is compact. Just to
get a feeling for what will come next, consider the case X = R.
Exercise A.9. Let µn be a sequence of probability measures on R. Prove
that the following are equivalent.
(a) µn has a weakly convergent subsequence converging to a probability
measure;
(b) for all ε > 0 there exist a < b such that sup n µn{[a, b]} > 1 − ε.
This necessary condition is in fact true more generally.
Definition A.10. Let X be a topological space. A family of probability
measures A is tight if for every ε > 0 there exists a compact subset K ⊂ X
such that sup ν∈A ν(K c ) < ε.
The following is a characterization of compactness of a family of probability
measures. See Theorems 6.1 and 6.2 of [6].
Prohorov’s theorem. Let X be a metric space. If a family of probability
measures is tight, then it is relatively sequentially compact (i.e. every
sequence has a weakly convergent subsequence). If X is separable and complete,
then the converse is also true.

170 A. Topics from probability
Note that unless the tight family is closed the limit point provided by
the theorem above does not necessarily belong to the same family.
A special case of Prohorov’s theorem is when A is a singleton.
∗ Exercise A.11. (Ulam’s theorem) Let X be separable and complete.
Prove, without using Prohorov’s theorem, that for any probability measure
λ ∈ M1(X ) and for any ε > 0 there exists a compact set K ⊂ X such
that λ(K c ) < ε.
Hint: By separability, there is a countable number of 1/n spheres (Ai,n)i≥1
covering X . Choose in such that λ{∪ in
i=1 Ai,n} > 1 − ε/2 n . The closure of
∩n≥1 ∪ in
i=1 Ai,n does the job.
The next theorem concerns M1(X ) being a Polish space (i.e. separable
and complete); see Theorem 6.5 in [30].
Theorem A.12. Suppose (X , d) is a complete separable metric space. Then
so is (M1(X ), ρ), the space of Borel probability measures on X with the
Prohorov metric defined by (A.2).
A.2. Ergodic theorem
Let (Ω, F , P ) be a probability space, d ≥ 1, and (θi) i∈Z d a group of measurable
P -preserving transformations of Ω such that θ0 is the identity and
θi ◦ θj = θi+j for all i, j ∈ Z d . Let I be the σ-algebra of all shift-invariant
events. (A is shift invariant if θiA = A for all i ∈ Z d .)
Exercise A.13. Prove that f : Ω → R is I-measurable if and only if
f ◦ θi = f for all i ∈ Z d .
Let Mθ(Ω) be the space of all shift-invariant probability measures on
(Ω, F ). Note that the example θk(ω) = ω + k on Ω = Z shows that this
space could be empty.
Definition A.14. P ∈ Mθ(Ω) is said to be ergodic if P (A) ∈ {0, 1} for all
I-measurable sets A.
Let (Vn)n≥1 be any increasing sequence of cubes in Z d such that |Vn| →
∞ as n → ∞. For ω ∈ Ω let
−1
Rn(ω) = |Vn|
i∈Vn
δθiω.
Multidimensional ergodic theorem. Setting as above. Let P ∈ Mθ(Ω).
Then, for all p ≥ 1 and f ∈ L p (P ), E Rn(ω) [f] → E[f | I] in L p (P ) and
P -a.s. When P is ergodic, the limit E[f | I] = E[f].

A.2. Ergodic theorem 171
To illuminate the theorem somewhat, note that for f ∈ L2 (P, F ) the
conditional expectation E[f | I] is the orthogonal projection of f onto the
space L2 (P, I). Thus the L2 version of the theorem says that the limit of
|Vn|
−1
i∈Vn f ◦ θi is the orthogonal projection of f onto the space of fixed
points of the transformations (θi). The L p version is obtained from this by
a density argument. The almost sure convergence is the hardest part. See
Appendix 14.A of [22] for the proof of the theorem.
A corollary of the ergodic theorem is that distinct ergodic measures are
supported on disjoint sets.
∗ Exercise A.15. Prove that if P1 and P2 are two different ergodic measures,
then they are mutually singular; i.e. there exists a measurable set A such
that P1(A) = 0 and P2(A) = 1.
Now, Mθ(Ω) is clearly a convex set. By definition, its extreme points
are measures P ∈ Mθ(Ω) such that P = tP1 + (1 − t)P2 with t ∈ (0, 1) and
P1, P2 ∈ Mθ(Ω) implies P1 = P2 = P . It turns out that the extreme points
are precisely the ergodic measures. This is again a corollary of the ergodic
theorem.
∗ Exercise A.16. A probability measure P ∈ Mθ(Ω) is ergodic if, and only
if, it is an extreme point of Mθ(Ω).
Hint: Use the above exercise.
Having a convex set it is natural to ask if there are enough extreme
points to recover the whole convex set by taking weighted averages. This
is indeed the case with the space Mθ(Ω) if, for example, (Ω, F ) is a Polish
space with its Borel sets.
Ergodic decomposition theorem. Setting as above. Assume additionally
that Ω is Polish and F is its Borel σ-algebra. Then, for any shift-invariant
measure P ∈ Mθ(Ω), there is a unique probability measure µP supported on
the set Me(Ω) of ergodic measures such that
P = Q µP (dQ).
Me(Ω)
The proof of the above theorem uses the existence of a conditional probability
Pω of P given I (see Example 8.5 for the definition of conditional
probability). The idea is to show that if P ∈ Mθ(Ω), then for P -a.e. choice
of ω, Pω is ergodic, and one can define µP as the distribution induced by
the map ω ↦→ Pω. See Exercises A.25 and A.26 at the end of this section for
the details.
For exercises A.17-A.24 we specialize to the following setting. (X , B) is
a measurable space and Ω = X Zd
is endowed with the product σ-algebra F .

172 A. Topics from probability
FV is the σ-algebra generated by {Xi : i ∈ V }. Vn is an increasing sequence
of cubes exhausting Z d . The tail σ-algebra is T = ∩nFV c n .
∗ Exercise A.17. (Kolmogorov’s 0-1 law) Let P be a product probability
measure on (Ω, F ); i.e. P = ⊗ i∈Z dλi, with λi ∈ M1(X ). Prove that P (A) ∈
{0, 1} for any T -measurable set A.
Hint: Observe that A is independent of FVn, for all n ≥ 1. Now prove that
the class of sets that are independent of A is a monotone class. Conclude
that A is independent of itself.
∗ Exercise A.18. Let P be a shift-invariant probability measure on Ω. Show
that if A is I-measurable, then there exists a set B that is T -measurable
and such that P (A∆B) = 0.
Hint: For each k ∈ N find Bk ∈ FV m(k) such that P (A∆Bk) < 2 −k ; see
Exercise 18 on page 32 of Folland’s textbook [18] for why such a Bk exists.
Check that B = ∪m≥n ∩k≥m θ −k−m(k)Bk works. In particular, note that B
is independent of n.
Exercise A.19. Let λ be a probability measure on X . Prove that λ⊗Zd is
ergodic for the shifts (θj) j∈Zd. In the next few exercises the space X will be a Polish space.
Exercise A.20. Suppose (X , d) is a separable metric space. Let Ω = X Zd
and let f : Ω → R be a measurable function. Prove that if f is continuous
I-measurable, then it is constant.
Hint: Let {xk} be a dense set in X . Find ˜ω ∈ Ω such that for any n and
any ω ∈ Ω such that ωi ∈ {xk} for all i ∈ Vn, there exists a j such that
˜ωj+Vn = ωVn. Prove that {θi˜ω : i ∈ Z d } is dense.
Exercise A.21. Prove that even though F is countably generated, I itself
is not.
Hint: Pick x = y ∈ X and let P = λ⊗Zd, with λ = (δx + δy)/2. If I
is generated by (Ak)k∈N, then let Bk = Ak if P (Ak) = 1 and Bk = Ac k if
P (Ak) = 0. Check that C = ∩Bk = {θiω : i ∈ Zd } for some ω ∈ Ω. Show
that P (C) should then be 0 and 1 at the same time.
Since distinct ergodic measures have disjoint supports, one can actually
define a universal version of the conditional probability Pω (given I)
simultaneously for all measures P ∈ Mθ(Ω).
Exercise A.22. Let X be a Polish space and define the set Ω0 = {ω :
κ(ω) = limn→∞ Rn(ω) exists}. For ω ∈ Ω0 define κ(ω) = Q for some fixed
Q ∈ Mθ(Ω). Prove that Ω0 is a Borel set and that κ : Ω → Mθ(Ω) is

A.2. Ergodic theorem 173
measurable. Moreover, κ ◦ θi = κ for all i ∈ Z d and for any bounded
measurable function f and bounded I-measurable function g,
(a) E κ(ω) [f] is I-measurable, and
(b) E P [gf] = g(ω)E κ(ω) [f]P (dω), for any P ∈ Mθ(Ω).
In other words, κ(ω) is a version of Pω, for any shift-invariant P .
Exercise A.23. Prove that the universal kernel κ from Exercise A.22 is
T -measurable.
Exercise A.24. Let P = λ⊗Zd for λ ∈ M1(X ). Let ˜ h(ω) = h(κ(ω) | P ),
where h is the specific relative entropy introduced in Section 7.2. Prove that
h(ν | P ) = h˜ dν, for all ν ∈ Mθ(Ω).
Hint: Use the affinity and lower semicontinuity of h together with the
ergodic decomposition ν = κ dν. If this is too difficult, Lemma 5.4.24 in
[9] gives a more general result.
The next exercise develops an abstract approach to the ergodic decomposition
from article [16]. The virtue of this approach is that it can be applied
to other examples too as illustrated by exercises. The key hypothesis is that
a collection of measures has a common kernel for conditional probabilities
relative to some σ-algebra. Exercise A.22 showed that such a kernel can be
defined for shift-invariant measures. Exercise A.27 below shows the same
for exchangeable measures, and Exercises 8.23 and 8.24 in Section 8.5 apply
Exercise A.25 to Gibbs measures.
∗ Exercise A.25. Let (Ω, F ) be a measurable space and M1 the space of
probability measures on (Ω, F ). Equip M1 with the smallest σ-algebra H
under which all functions µ ↦→ g dµ are measurable for g ∈ bF . Fix a
nonempty measurable subset P of M1 and a sub-σ-algebra A of F . Define
the subset of A -ergodic measures by
Make two assumptions.
(A.3)
Pe = {µ ∈ P : µ(A) = 0 or 1 ∀A ∈ A }.
(i) There exists a countable family W ⊂ bF of bounded measurable
functions that distinguishes elements of P: if µ, ν ∈ P and g dµ =
g dν for all g ∈ W then µ = ν.
(ii) There exists a stochastic kernel κ from (Ω, A ) into (Ω, F ) with
these properties:
κ(ω, · ) ∈ P for all ω ∈ Ω.
For every µ ∈ P and B ∈ F , κ(ω, B) is a version of µ(B | A )(ω).
(a) Let µ ∈ P. Show that statements (i)–(iv) are equivalent.

174 A. Topics from probability
(i) µ ∈ Pe.
(ii) Eκ(ω) [g] = E µ [g] for µ-a.e. ω, for all g ∈ W .
(iii) Φg(µ) = 0 for all g ∈ W , where
E κ(ω) µ
Φg(µ) = [g] − E [g] µ(dω) =
2
(iv) µ{ω : κ(ω) = µ} = 1.
Hint: (i)=⇒(ii)=⇒(iii)=⇒(iv)=⇒(i).
E κ(ω) [g] 2 µ(dω) − E µ [g] 2 .
(b) Establish these statements.
(i) Pe is a measurable subset of M1, and a subset of the image
{κ(ω) : ω ∈ Ω}.
(ii) For any µ ∈ P, µ{ω : κ(ω) ∈ Pe} = 1.
(iii) Pe is not empty.
Hint: Pe is the intersection of the sets {Φg = 0}. By conditioning
on A derive Φg(κ(ω)) µ(dω) = 0.
(c) Show that for every µ ∈ P there exists a unique probability measure
Qµ on Pe such that
µ(B) =
Pe
ν(B) Qµ(dν) ∀B ∈ F .
Hint: Let Qµ(D) = µ{ω : κ(ω) ∈ D}, after adjusting κ(ω) so that
all its values are in Pe. For uniqueness, use point (iv) of part (a)
to show that if Qµ gives the decomposition then it must be the
distribution of κ(ω) under µ.
∗ Exercise A.26. (Ergodic decomposition) Apply Exercise A.25 to deduce
the ergodic decomposition. Define the kernel κ(ω) as in Exercise A.22.
∗ Exercise A.27. (de Finetti’s theorem) Let S be a Polish space and Ω =
S N the space of S-valued sequences with its product σ-algebra F . Let
Sn denote the group of permutations (bijective mappings) on {1, 2, . . . , n}.
Permutations π ∈ Sn act on Ω in the obvious way: (πω)i = ω π(i) where we
take π(i) = i for i > n. Define σ-algebras
and the exchangeable σ-algebra
En = {A ∈ F : π −1 A = A ∀π ∈ Sn}
E =
En.
n≥1
A probability measure µ ∈ M1(Ω) is exchangeable if it is invariant under
all finite permutations.
The goal here is to apply Exercise A.25 to prove the following.

A.3. Stochastic ordering 175
de Finetti’s theorem. Fix an exchangeable measure µ. Then, conditional
on E , the coordinates Xi(ω) = ωi are i.i.d. under µ, and µ can be decomposed
into a mixture of i.i.d. measures.
Define the empirical measures
An(ω) = 1
n!
π∈Sn
δπω.
(a) Show that E An(ω) [f] = E µ [f | En](ω) and consequently, by the backwards-martingale
convergence theorem (Theorem 6.1 on page 265
of [15] or page 155 of [26]), E An [f] → E µ [f | E ] µ-a.s. Show that
out of these limits we can construct a kernel that satisfies (A.3).
(b) Let f be a bounded function on S k and g a bounded function on
S. Show that
E An f(X1, . . . , Xk)g(Xk+1) − E An f(X1, . . . , Xk) E An g(X1) = O(n −1 ).
Show that any limit point of An is an i.i.d. product measure.
(c) Show that E is trivial under an exchangeable measure µ if and only
if µ is an i.i.d. product measure. Note that one direction here is the
Hewitt-Savage 0-1 law. Complete the proof of de Finetti’s theorem.
A.3. Stochastic ordering
Strassen’s lemma. Let Ω = {−1, 1} Zd.
Let µ, ν ∈ M1(Ω) be such that
µ ≤ ν. Assume that µ{σi = 1} = ν{σi = 1} for all i ∈ Zd . Then µ = ν.
Proof. We will prove the following claim for all N ≥ 1. Recall that ΩΛ =
{−1, 1} Λ .
Claim A.28. For any Λ ⊂ Z d with |Λ| = N and µ, ν ∈ M1(ΩΛ), if µ ≤ ν
and µ(σi = 1) = ν(σi = 1) for all i ∈ Λ, then µ = ν.
This will prove that the marginals of the µ and ν in the statement of
the lemma coincide and thus µ = ν.
To prove the claim we will use induction over N. The case N = 1 is
obvious. The case N = 2 can be done by direct computation. Indeed, if
Λ = {i, j}, then observe that 1I{σi = 1 or σj = 1} is increasing. Thus,
µ{σi = −1, σj = 1} + µ{σi = 1, σj = 1} + µ{σi = 1, σj = −1}
≤ ν{σi = −1, σj = 1} + ν{σi = 1, σj = 1} + ν{σi = 1, σj = −1}.
But the sums of the first two terms on each side are equal. Thus,
µ{σi = 1, σj = −1} ≤ ν{σi = 1, σj = −1}.

176 A. Topics from probability
Since 1I{σi = 1, σj = 1} is increasing we also have
µ{σi = 1, σj = 1} ≤ ν{σi = 1, σj = 1}.
The sum of the left-hand-sides of the above two inequalities equals that of the
right-hand-sides. Thus, these must actually be equalities. Interchanging the
roles of i and j also proves that µ{σi = −1, σj = 1} = ν{σi = −1, σj = 1}.
Then µ{σi = −1, σj = −1} = ν{σi = −1, σj = −1} follows, proving the
claim when N = 2.
Now assume the claim is true for N spins, with N ≥ 2. Consider measures
on ΩΛ with |Λ| = N + 1. By Exercise 10.12, if µ and ν agree on all
events of the type K(A) = {σi = 1 ∀i ∈ A}, A ⊂ Λ, then µ = ν. So suppose
they do not agree on all such events. Since their N-spin marginals do agree,
the only case of possible disagreement is A = Λ.
So suppose there is ε > 0 such that ν{K(Λ)} = µ{K(Λ)} + ε. (Since
µ ≤ ν the difference must go this way.)
Again since the marginals on N spins agree, it must be that for any
i ∈ Λ, µ{K(Λ {i}), σi = −1} = ν{K(Λ {i}), σi = −1} + ε.
However, if now we take i, j ∈ Λ, with i = j, and set
B = K(Λ {i, j}) ∩ ({σi = 1} ∪ {σj = 1}),
then from above µ(B) = ν(B) + 2ε − ε = ν(B) + ε. This is a contradiction
with µ ≤ ν because 1IB is an increasing function.

Topics from analysis
Ultimately, this will be an appendix on general analysis stuff.
B.1. Measure-theoretic lemma
Appendix B
Lemma B.1. Let X be a metric space and let H be a class of bounded
functions that contains the space Ub(X ) of bounded uniformly continuous
functions and is closed under uniformly bounded pointwise convergence (i.e.
fn ∈ H for all n, maxn sup x |fn(x)| < ∞, and fn(x) → f(x) for all x ∈ X
implies f ∈ H). Then bB ⊂ H.
For this we need another technical lemma.
Lemma B.2. Let X be a metric space and let H be a class of bounded
functions that contains the space Ub(X ) of bounded uniformly continuous
functions and is closed under uniformly bounded pointwise convergence. Fix
an arbitrary function g. Suppose g + f ∈ H for all f ∈ Ub(X ). Then
g + α1IA + f ∈ H for all real α, all f ∈ Ub(X ), and all Borel sets A.
Proof. Let
C = {A ⊂ X : g + α1IA + f ∈ H, ∀α ∈ R, ∀f ∈ Ub(X )}.
C contains the algebra
A = {A ⊂ X : ∃fn ∈ Ub(X ) uniformly bounded : 1IA = lim
n→∞ fn}
because if A ∈ A then
g + α1IA + f = lim
n→∞ (g + αfn + f) ∈ H
177

178 B. Topics from analysis
by the hypothesis and closedness under uniformly bounded pointwise limits.
C is also a monotone class because if Ak ∈ C and 1IAk → 1IA then
g + α1IA + f = lim (g + α1IAk + f) ∈ H.
k→∞
By the monotone class theorem (see page of 30 of [26]) C contains the
σ-algebra generated by A. This is all of B because A contains all open
and closed sets. (Recall that if A is closed, 1IA is approximated by fn(x) =
(1 + nd(x, A)) −1 , where d is the metric on X .)
Proof of Lemma B.1. The hypothesis of the above lemma is true for g =
0. Consequently α1IA + f ∈ H for all f ∈ Ub(X ), α ∈ R, and A measurable.
Suppose we have shown that H contains all functions of the form h + f
where h is a simple function with at most n terms and f ∈ Ub(X ). (The case
n = 1 has been verified in the previous paragraph.) Then the above lemma
shows again that H contains also all h + f where h is a simple function with
at most n + 1 terms and f ∈ Ub(X ). By taking f = 0 we get that all simple
functions are in H. Since bounded measurable functions are uniform limits
of simple ones, H contains all bounded measurable functions.
B.2. Minimax theorem
In this section we will prove a theorem of König [27] following the proof
given by Kassay [25].
Let X and Y be nonempty sets and f : X × Y → R a given function.
We say f is uniformly Jensen-concave-convex-like if
(B.1)
and
(B.2)
∀y0, y1 ∈ Y, ∃y ∈ Y : ∀x ∈ X , f(x, y) ≤ (f(x, y0) + f(x, y1))/2,
∀x0, x1 ∈ X , ∃x ∈ X : ∀y ∈ Y, f(x, y) ≥ (f(x0, y) + f(x1, y))/2.
Of course, if X and Y are convex sets and f(x, y) is concave in x for
each fixed y, and convex in y for each fixed x, then f is also uniformly
Jensen-concave-convex-like.
Let D be the set of dyadic rational numbers in [0, 1]; i.e. numbers of the
form k/2 n for some integers n ≥ 1 and k ∈ {1, . . . , 2 n }.
Exercise B.3. Show that (B.1) implies that for every y0, y1 ∈ Y and t ∈ D,
there exists yt ∈ Y such that we have for all x ∈ X ,
f(x, yt) ≤ (1 − t)f(x, y0) + tf(x, y1).
Similarly, (B.2) implies that for every x0, x1 ∈ X and t ∈ D there exists
xt ∈ X such that we have for all y ∈ Y,
f(xt, y) ≥ (1 − t)f(x0, y) + tf(x1, y).

B.2. Minimax theorem 179
∗ Exercise B.4. Fix an integer k ≥ 2. Show that there exists a dense set
Mk ⊂
(t1, . . . , tk) ∈ [0, 1] k :
k
i=1
such that for t = (t1, . . . , tk) ∈ Mk the following hold:
ti = 1
(a) for every x1, . . . , xk ∈ X , there exists xt ∈ X such that for every
y ∈ Y
k
f(xt, y) ≥ tif(xi, y).
i=1
(b) for every y1, . . . , yk ∈ Y, there exists yt ∈ X such that for every
x ∈ X
k
f(x, yt) ≤ tif(x, yi).
i=1
König’s minimax theorem. Suppose X is a compact Hausdorff space and
f : X × Y → R is uniformly Jensen-concave-convex-like. Assume f(·, y) :
X → R is upper semicontinuous for every fixed y ∈ Y. Then
sup
x∈X
inf f(x, y) = inf
y∈Y y∈Y sup f(x, y).
x∈X
Proof. One direction is easy. Indeed, it is always true that
sup
x∈X
inf f(x, y) ≤ inf
y∈Y y∈Y sup f(x, y).
The other direction needs the following lemma. Let c + = infy sup x f(x, y)
and for c < c + and y ∈ Y define the (nonempty) set
x∈X
Hc,y = {x ∈ X : f(x, y) ≥ c}.
Lemma B.5. Same assumptions as König’s minimax theorem. Then, for
each c < c + , y1, . . . , yn ∈ Y, we have ∩n Hc,yi i=1 = ∅.
Proof. Fix c < c + and assume ∩n Hc,yi i=1 = ∅. Define the function h : X →
Rn by
h(x) = (f(x, y1) − c, . . . , f(x, yn) − c)
and set K = [0, ∞) n . Let co(h(X )) be the closure of the convex hull of
h(X ); i.e. the intersection of all convex closed sets containing h(X ). Let K ◦
be the interior of K.
If co(h(X )) ∩ K ◦ = ∅, then there must exist s1, . . . , sk ∈ [0, 1] with
k
i=1 si = 1 and x1, . . . , xk ∈ X such that k
i=1 sih(xi) ∈ K ◦ . Recall
Exercise B.4. Choose (t1, . . . , tk) ∈ Mk such that k
i=1 tih(xi) ∈ K. Then,
h(xt) − k
i=1 tih(xi) is also in K, which implies that h(X ) ∩ K = ∅ and
contradicts the assumption ∩ n i=1 Hc,yi = ∅. Thus, co(h(X )) ∩ K◦ = ∅.

180 B. Topics from analysis
h₁(χ)
p₂
p₁
t‧u=0
δ‧u=-d
Figure B.1. The hyperplane separating h1(X ) and K and passing
through the origin.
By the Hahn-Banach separation theorem (page 42), there exists a hyperplane
which separates co(h(X )) and K◦ ; i.e. there exist γ and α ∈ Rn ,
such that for all x ∈ X and u ∈ K◦ ,
n
n
αiui < γ ≤ αi(f(x, yi) − c).
i=1
i=1
Taking ui → ∞ shows that αi ≤ 0. Taking u → 0 shows that γ ≥ 0. Clearly,
there exists an i ≤ n such that αi = 0. Setting δ = α/ n i=1 αi ∈ [0, 1] n one
has n i=1 δi = 1 and n i=1 δif(x, yi) ≤ c.
Now take c < c1 < c + and set d = c1 − c. Then, for all x ∈ X ,
n
δi[f(x, yi) − c1] ≤ −d.
i=1
Hence, if h1(x) = (f(x, y1) − c1, . . . , f(x, yn) − c1), then h1(X ) is separated
from K by the hyperplane n
i=1 δiui = −d. Since X is compact and f(·, yi)’s
are upper semicontinuous, they are bounded above; see Exercise 2.9. Thus,
there exists p ∈ K ◦ such that h1(X ) ⊂ n
i=1 (−∞, pi]. One then can find
a t ∈ Mn (recall Exercise B.4) such that the hyperplane n
i=1 tiui = 0 still
separates h1(X ) and K; see Figure B.1. This means that
n
tif(x, yi) ≤ c1
i=1
for all x ∈ X . Consider now yt ∈ Y such that f(x, yt) ≤ n i=1 tif(x, yi) for
all x ∈ X . Then, c + ≤ supx∈X f(x, yt) ≤ c1. This contradicts c1 < c + and
proves that ∩n Hc,yi i=1 = ∅.

B.2. Minimax theorem 181
Now, if ∩yHc,y = ∅, then {X Hc,y; y ∈ Y} is a family of open sets
covering X . Since X is compact, it admits a finite covering. But this would
imply the existence of finitely many sets Hc,y with empty intersection. Thus,
∩yHc,y = ∅ and sup x infy f(x, y) ≥ c. The theorem follows from taking
c ↗ c + .

Inequalities
C.1. Holley’s inequality
Appendix C
Define pointwise minima and maxima by (η∧ξ)i = ηi∧ξi and (η∨ξ)i = ηi∨ξi.
Holley’s inequality. Let ΩΛ = {−1, 1} Λ for some finite Λ ⊂ Z d . Let µ
and ν be strictly positive probability measures on ΩΛ. If
(C.1)
for all η and ξ in ΩΛ, then µ ≤ ν.
µ(η ∧ ξ)ν(η ∨ ξ) ≥ µ(η)ν(ξ)
Proof. The idea is to couple two Markov chains on ΩΛ = {−1, 1} Λ with
stationary distributions µ and ν so that the coupled process is a Markov
chain on the space {(η, ξ) ∈ ΩΛ × ΩΛ : η ≤ ξ}.
Let us use the notation η j for the configuration where the j-th spin in η
is flipped. That is, η j
i = ηi if i = j and η j
j = −ηj. At each step the Markov
chain will choose a site in Λ and flip the η-spin and/or the ξ-spin at that
site. To define the transition probabilities set first for each j ∈ Λ
r((η, ξ), (η j , ξ)) = 1 if ηj = −1 and ξj = 1,
r((η, ξ), (η, ξj )) = ν(ξj )
ν(ξ)
r((η, ξ), (η
if ηj = −1 and ξj = 1,
j , ξj )) = 1 if ηj = ξj = −1,
r((η, ξ), (ηj , ξj )) = ν(ξj )
ν(ξ)
r((η, ξ), (η
if ηj = ξj = 1,
j , ξ)) = µ(ηj )
µ(η) − ν(ξj )
ν(ξ)
r((η, ξ), (η
if ηj = ξj = 1, and
′ , ξ ′ )) = 0 otherwise.
183

184 C. Inequalities
Next, let
C = max
(η,ξ)
(η ′ ,ξ ′ )
and define the transition probabilities
when (η, ξ) = (η ′ , ξ ′ ) and
r((η, ξ), (η ′ , ξ ′ )),
p((η, ξ), (η ′ , ξ ′ )) = r((η, ξ), (η ′ , ξ ′ ))/C,
p((η, ξ), (η, ξ)) = 1 − C
−1
r((η, ξ), (η ′ , ξ ′ )).
The definition above requires µ(η j )ν(ξ) ≥ µ(η)ν(ξ j ) whenever ηj = ξj =
1 which is ensured by condition (C.1). These transition probabilities preserve
η ≤ ξ.
Exercise C.1. Prove that each of the two coordinates of the above Markov
chain is itself a Markov chain. Show that the transition probabilities of the
η-chain are
p(η, η j ) = 1/C if ηj = −1,
p(η, η j ) = µ(ηj )
Cµ(η)
η ′ ,ξ ′
if ηj = 1, and
p(η, η) = 1 −
j p(η, ηj ) for all η.
Also show that the transition probabilities of the ξ-chain are
p(ξ → ξ j ) = 1/C if ξj = −1,
p(ξ, ξ j ) = ν(ξj )
Cν(ξ)
if ξj = 1, and
p(ξ, ξ) = 1 −
j p(ξ, ξj ) for all ξ.
Both chains are clearly irreducible, since one can obtain any configuration
from any other one by a finite number of spin flips.
Exercise C.2. Show that µ is the invariant measure for the η-chain (denoted
by η(n)) and ν is the invariant measure for the ξ-chain (denoted by
ξ(n)).
Thus, if f is an increasing function on Ω, then
E[f(η(n)) | η(0) = η] = E[f(η(n)) | η(0) = η, ξ(0) = ξ]
≤ E[f(ξ(n)) | η(0) = η, ξ(0) = ξ]
= E[f(ξ(n)) | ξ(0) = ξ],
whenever η ≤ ξ. As n → ∞ the inequality E µ [f] ≤ E ν [f] follows by the
Markov chain convergence theorem (Theorem 5.5 on page of 314 of [15] or
Exercise 6.17).

C.2. Griffiths’ inequality 185
C.2. Griffiths’ inequality
Griffiths’ inequality. Fix a finite volume Λ ⊂ Zd . Let E = {{i, j} : |i−j| =
1, i ∈ Λ} be the set of nearest-neighbor edges of Λ, including edges from Λ
to its complement. Define the function F : [0, ∞) E → [−1, 1] by
σΛ
F (J) =
σ0
exp {i,j}∈E Ji,j
σiσj
exp {i,j}∈E Ji,j
,
σiσj
σΛ
where J = (Ji,j) {i,j}∈E, the sum runs over σΛ ∈ {−1, 1} Λ , and for j ∈ Λ we
set σj = 1 (i.e. the + boundary condition). Then,
∂F
∂Ji,j
(J) ≥ 0 ∀{i, j} ∈ E and ∀J ∈ [0, ∞) E .
We start by proving the following lemma.
Lemma C.3. Assumptions as above. For any J ∈ [0, ∞) E and {k, ℓ} ∈ E,
σ0σkσℓ exp
≥ 0.
σΛ
{i,j}∈E
Ji,j σiσj
Proof. The proof goes by a direct computation, expanding the exponential.
σ0σkσℓ exp
σΛ
=
σΛ
=
n≥0
{i,j}∈E
σ0σkσℓ
n≥0
1
n!
Ji,j σiσj
1
n!
{i,j}∈E
{ik,jk}∈E, k=1,...,n
n
Ji,j σiσj
k=1
Jik,jk
n
σΛ
σ0σkσℓ
n
k=1
σik σjk
The expression in square brackets vanishes if some spin inside Λ appears in
the product an odd number of times. Otherwise the product contains only
squares and (+1)-valued boundary spins and hence is positive.
Proof of Griffiths’ inequality. A direct computation shows that
∂F
−2
= Z σ0(σkσℓ − τkτℓ) exp
Ji,j(σiσj + τiτj) ,
∂Jk,ℓ
where Z =
σΛ
σΛ,τΛ
exp
{i,j}∈E Ji,j σiσj
{i,j}∈E
and, just as for σΛ, τΛ ranges over
{−1, 1} Λ and τi = 1 when i ∈ Λ. Define the configuration η such that ηi = 1
if σi = τi and −1 if σi = τi. Noting that τi = ηiσi for all i (in particular,
ηi = 1 when i ∈ Λ), a change of variables gives
∂F
∂Jk,ℓ
= Z
−2
ηΛ
(1 − ηkηℓ)
σ0σkσℓ exp
σΛ
{i,j}∈E
Ji,j(1 + ηiηj)σiσj
.
.

186 C. Inequalities
The inequality now follows from the above lemma using the couplings Ji,j(1+
ηiηi) ≥ 0 instead of Ji,j.
C.3. Griffiths-Hurst-Sherman inequality
Griffiths-Hurst-Sherman inequality. Consider the Ising model in d ≥ 1.
Fix β > 0, J > 0, and the volume Vn. Define
Mn(h) = E µ+
1
Vn,β,h,J σi .
|Vn|
Then ∂2 Mn
∂h 2 ≤ 0 for h > 0.
First, we need to prove two lemmas. Let us abbreviate µ +
n,h = µ+
Vn,β,h,J
and let (µ +
n,h )⊗4 be the product of 4 copies of µ +
n,h . We will denote the
elements of Ω4 Vn by (ωi, σi, ω ′ i , σ′ i )i∈Vn.
Lemma C.4. Same setting as for the Griffiths-Hurst-Sherman inequality.
For i ∈ Vn, let
⎡ ⎤ ⎡
αi
⎢βi⎥
⎢ ⎥
1 ⎢
⎣γi
⎦ = ⎢
2 ⎣
1
1
1
1
1
−1
1
−1
1
⎤ ⎡ ⎤
1 ωi
−1⎥
⎢σi⎥
⎥ ⎢ ⎥
−1⎦
⎣ ⎦
−1 1 1 −1
.
δi
i∈Vn
Let A, B, C, and D be arbitrary subsets of Vn. If h ≥ 0, then
n,h )⊗4
(C.2)
≥ 0.
E (µ+
i∈A
αi
i∈B
βi
Proof of Lemma C.4. Since the above matrix is orthonormal, it preserves
Euclidian scalar products and thus
i∈C
γi
ω ′ i
σ ′ i
i∈D
ωiωj + σiσj + ω ′ iω ′ j + σ ′ iσ ′ j = αiαj + βiβj + γiγj + δiδj,
for any i, j ∈ Vn. Therefore,
H + n (ω) + H + n (σ) + H + n (ω ′ ) + H + n (σ ′ )
= −J
i,j∈Vn:|i−j|=1
(αiαj + βiβj + γiγj + δiδj) − 2J
δi
i∈Vn,j∈Vn
|i−j|=1
αi − 2h
αi.
Now we proceed similarly to the proof of Lemma C.3. Expand the term
e −β(H+ n (ω)+H + n (σ)+H + n (ω ′ )+H + n (σ ′ )) into its Taylor series in the numerator of the
expectation in (C.2). This leads us to sums of products of multiple integrals
of the form
α k i β ℓ i γ m i δ n i λ(dωi)λ(dσi)λ(dω ′ i)λ(dσ ′ i),
multiplied by nonnegative coefficients.
i∈Vn

C.3. Griffiths-Hurst-Sherman inequality 187
∗Exercise C.5. Check that αiβi = γiδi = (ωiσi−ω ′ iσ′ i )/2 and then conclude
that αiβiγiδi = (ωiσi − ω ′ iσ′ i )2 /4.
∗ Exercise C.6. Prove that the above integral is 0 if k, ℓ, m, and n are not
all of the same parity (i.e. all even or all odd).
By the above exercise we can assume that the powers are all of the same
parity. If the powers are all even the integral is obviously nonnegative. If the
powers are all odd use Exercise C.5 to show that the integral is nonnegative.
Lemma C.7. Same setting as for the Griffiths-Hurst-Sherman inequality.
Let i, j, k ∈ Vn be arbitrary. Abbreviate ¯σℓ = E µ+
n,h[σℓ]. If h ≥ 0, then
(C.3)
Proof. Let
E µ+ n,h[(σi − ¯σi)(σj − ¯σj)(σk − ¯σk)] ≤ 0,
⎡
⎢
⎣
ti
qi
t ′ i
q ′ i
⎤
⎥
⎦
= 1
√ 2
⎡
1 1 0
⎤ ⎡
0
⎢
⎢1
⎣0
−1
0
0
1
0⎥
⎢
⎥ ⎢
1⎦
⎣
0 0 1 −1
ωi
σi
ω ′ i
σ ′ i
⎤
⎥
⎦ .
Exercise C.8. Check that the left-hand-side of (C.3) is equal to
− √ 2E (µ+ n,h )⊗4
[tkq ′ iq ′ j − tkqiqj] = −E (µ+ n,h )⊗4
[(αk + βk)(γiδj + γjδi)],
where the functions αi, βi, γi, and δi are as in the previous lemma.
Hint: The first equality is a direct computation. For the second equality,
write tk = (αk + βk)/ √ 2, q ′ i = (γi + δi)/ √ 2, and qi = (γi − δi)/ √ 2.
The claim then follows from the previous lemma.
Proof of the Griffiths-Hurst-Sherman inequality. Set Sn =
i∈Vn σi
and ¯ Sn =
i∈Vn ¯σi. The above lemma shows that if h ≥ 0, then
E µ+
n,h[S 3 n] − 3E µ+
n,h[Sn]E µ+
n,h[S 2 n] + 2E µ+
n,h[Sn] 3 = E µ+ n,h[(Sn − ¯ Sn) 3 ] ≤ 0.
The inequality follows now from the following computation.
∗ Exercise C.9. Show that
β −2 |Vn| ∂2Mn = Eµ+ n,h[S
∂h2 3 n] − 3E µ+
n,h[Sn]E µ+
n,h[S 2 n] + 2E µ+
n,h[Sn] 3 .

Bibliography
1. Michael Aizenman, Instability of phase coexistence and translation invariance
in two dimensions, Mathematical problems in theoretical
physics (Proc. Internat. Conf. Math. Phys., Lausanne, 1979), Lecture
Notes in Phys., vol. 116, Springer, Berlin, 1980, pp. 143–147. MR582616
2. , Translation invariance and instability of phase coexistence in
the two-dimensional Ising system, Comm. Math. Phys. 73 (1980), no. 1,
83–94. MR573615
3. Robert G. Bartle, The elements of real analysis, first corrected printing
ed., John Wiley & Sons, New York-London-Sydney, 1967. MR0393369
4. J. R. Baxter, N. C. Jain, and T. O. Seppäläinen, Large deviations for
nonstationary arrays and sequences, Illinois J. Math. 37 (1993), no. 2,
302–328. MR1208824
5. John Baxter and Naresh Jain, Convexity and compactness in large deviation
theory, Unpublished manuscript (1991).
6. Patrick Billingsley, Convergence of probability measures, John Wiley &
Sons Inc., New York, 1968. MR0233396
7. J. Bricmont, J. R. Fontaine, and L. J. Landau, On the uniqueness of
the equilibrium state for plane rotators, Comm. Math. Phys. 56 (1977),
no. 3, 281–296. MR0489629
8. Amir Dembo and Ofer Zeitouni, Large deviations techniques and applications,
second ed., Applications of Mathematics, vol. 38, Springer-Verlag,
New York, 1998. MR1619036
9. Jean-Dominique Deuschel and Daniel W. Stroock, Large deviations,
Pure and Applied Mathematics, vol. 137, Academic Press Inc., Boston,
MA, 1989. MR997938
189

190 Bibliography
10. I. H. Dinwoodie, A note on the upper bound for i.i.d. large deviations,
Ann. Probab. 19 (1991), no. 4, 1732–1736. MR1127723
11. R. L. Dobrushin, Existence of a phase transition in the two-dimensional
and three-dimensional Ising models, Soviet Physics Dokl. 10 (1965),
111–113. MR0182405
12. , Gibbsian random fields for lattice systems with pairwise interactions.,
Funkcional. Anal. i Priloˇzen. 2 (1968), no. 4, 31–43. MR0250630
13. , The Gibbs state that describes the coexistence of phases for a
three-dimensional Ising model, Teor. Verojatnost. i Primenen. 17 (1972),
619–639. MR0421546
14. Richard M. Dudley, Real analysis and probability, The Wadsworth &
Brooks/Cole Mathematics Series, Wadsworth & Brooks/Cole Advanced
Books & Software, Pacific Grove, CA, 1989. MR982264
15. Richard Durrett, Probability: theory and examples, second ed., Duxbury
Press, Belmont, CA, 1996. MR1609153
16. E. B. Dynkin, Initial and final behavior of the trajectories of Markov processes,
Uspehi Mat. Nauk 26 (1971), no. 4(160), 153–172. MR0298758
17. William Feller, An introduction to probability theory and its applications.
Vol. I, Third edition, John Wiley & Sons Inc., New York, 1968.
MR0228020
18. Gerald B. Folland, Real analysis: Modern techniques and their applications,
second ed., Pure and Applied Mathematics (New York), John
Wiley & Sons Inc., New York, 1999. MR1681462
19. Klaus Fritzsche and Hans Grauert, From holomorphic functions to complex
manifolds, Graduate Texts in Mathematics, vol. 213, Springer-
Verlag, New York, 2002. MR1893803
20. Jürg Fröhlich and Charles-Edouard Pfister, Spin waves, vortices, and
the structure of equilibrium states in the classical XY model, Comm.
Math. Phys. 89 (1983), no. 3, 303–327. MR709469
21. Jürg Fröhlich and Thomas Spencer, Kosterlitz-Thouless transition in
the two-dimensional plane rotator and Coulomb gas, Phys. Rev. Lett.
46 (1981), no. 15, 1006–1009. MR607429
22. Hans-Otto Georgii, Gibbs measures and phase transitions, de Gruyter
Studies in Mathematics, vol. 9, Walter de Gruyter & Co., Berlin, 1988.
MR956646
23. Robert B. Griffiths, Peierls proof of spontaneous magnetization in a twodimensional
Ising ferromagnet, Phys. Rev. (2) 136 (1964), A437–A439.
MR0189681

Bibliography 191
24. Y. Higuchi, On the absence of non-translation invariant Gibbs states
for the two-dimensional Ising model, Random fields, Vol. I, II (Esztergom,
1979), Colloq. Math. Soc. János Bolyai, vol. 27, North-Holland,
Amsterdam, 1981, pp. 517–534. MR712693
25. G. Kassay, A simple proof for König’s minimax theorem, Acta Math.
Hungar. 63 (1994), no. 4, 371–374. MR1261480
26. Davar Khoshnevisan, Probability, Graduate Studies in Mathematics,
vol. 80, American Mathematical Society, Providence, RI, 2007.
MR2296582
27. Heinz König, Über das von Neumannsche Minimax-Theorem, Arch.
Math. (Basel) 19 (1968), 482–487. MR0240600
28. O. E. Lanford, III and D. Ruelle, Observables at infinity and states with
short range correlations in statistical mechanics, Comm. Math. Phys.
13 (1969), 194–215. MR0256687
29. A. Messager, S. Miracle-Sole, and C. Pfister, Correlation inequalities and
uniqueness of the equilibrium state for the plane rotator ferromagnetic
model, Comm. Math. Phys. 58 (1978), no. 1, 19–29. MR0475552
30. K. R. Parthasarathy, Probability measures on metric spaces, AMS
Chelsea Publishing, Providence, RI, 2005, Reprint of the 1967 original.
MR2169627
31. Robert R. Phelps, Lectures on Choquet’s theorem, second ed., Lecture
Notes in Mathematics, vol. 1757, Springer-Verlag, Berlin, 2001.
MR1835574
32. R. Tyrrell Rockafellar, Convex analysis, Princeton Mathematical Series,
No. 28, Princeton University Press, Princeton, N.J., 1970. MR0274683
33. Walter Rudin, Functional analysis, second ed., International Series in
Pure and Applied Mathematics, McGraw-Hill Inc., New York, 1991.
MR1157815
34. Erwin Schrödinger, Statistical thermodynamics, A course of seminar
lectures delivered in January-March 1944, at the School of Theoretical
Physics, Dublin Institute for Advanced Studies. Second edition,
reprinted, Cambridge University Press, New York, 1962. MR0149891
35. Timo Seppäläinen, Large deviations for lattice systems. I. Parametrized
independent fields, Probab. Theory Related Fields 96 (1993), no. 2, 241–
260. MR1227034
36. , Large deviations for lattice systems. II. Nonstationary independent
fields, Probab. Theory Related Fields 97 (1993), no. 1-2, 103–112.
MR1240718

192 Bibliography
37. , Entropy, limit theorems, and variational principles for disordered
lattice systems, Comm. Math. Phys. 171 (1995), no. 2, 233–277.
MR1344727
38. , Maximum entropy principles for disordered spins, Probab. Theory
Related Fields 101 (1995), no. 4, 547–576. MR1327225
39. Timo Seppäläinen and J. E. Yukich, Large deviation principles for Euclidean
functionals and other nearly additive processes, Probab. Theory
Related Fields 120 (2001), no. 3, 309–345. MR1843178
40. H. van Beijeren, Interface sharpness in the Ising system, Comm. Math.
Phys. 40 (1975), no. 1, 1–6. MR1552609

Notation index
empirical mean (X1 + · · · + Xn)/n
Sn
[x] integral part of x; i.e. largest integer smaller or equal to x
an ∼ bn an is equivalent to bn; i.e. an/bn → 1
BER(p) Bernoulli distribution with parameter p
N set of positive integers
Z set of whole numbers
Q set of rational numbers
R set of real numbers
a.s. almost surely
a.e. almost every
i.i.d. independent, identically distributed
E[f] expectation of f, relative to P
E µ [f] expectation of f, relative to some measure µ
X , Y topological spaces (Hausdorff, metric, vector, Polish, etc)
B Borel σ-algebra of the topological space X
Ω a general probability space and often Ω = X Zd
F σ-algebra on Ω and the product Borel σ-algebra if Ω = X Zd
S Polish space
M1(X ) probability measures on X
M(X ) finite signed measures on X
A◦ interior of A
A closure of A
Ac compliment of A
A B set difference
A∆B symmetric set difference (A B) ∪ (B A)
|A| cardinality of A
193

194 Notation index
lim limsup
lim liminf
flsc
lower semicontinuous regularization of f
LDP(µn, rn, I) {µn} satisfy a large deviation principle with
normalization rn and rate function I
ν ≪ λ ν is absolutely continuous relative to λ
∇f
Ln
gradient of f
empirical measures: 1 n n k=1 δXk
Cb(X )
C
bounded continuous functions on X
+
b (X ) functions in Cb(X ) that are strictly positive and
bounded away from 0
bB bounded B-measurable functions
〈·, ·〉 bilinear duality between two vector spaces
f ∗ convex conjugate of f
f ∗∗ convex biconjugate of f
∂f subdifferential of f
p(·) pressure function
Ub,d(X ) bounded uniformly continuous functions on (X , d)
Ub(X ) bounded uniformly continuous functions on X
H(ν | λ) entropy of ν relative to λ
β inverse temperature
λ⊗N the law of an i.i.d. sequence with marginals λ
λ⊗Zd the law of an i.i.d. field with marginals λ
θi
group of shifts on a probability space Ω
Mθ(Ω) shift-invariant probability measures on Ω
Me(Ω) ergodic probability measures on Ω
T tail σ-algebra
I shift-invariant σ-algebra
ωΛ
ΩΛ
spins in Λ: (ωi)i∈Λ
space X Λ of spins in Λ
FΛ
Rn
σ-algebra generated by spins in Λ
level-3 empirical fields: Rn(ω) = 1
|Vn| i∈Vn δθiω
Cb,loc(Ω) bounded continuous local functions
Ub,loc(Ω) bounded uniformly continuous local functions
f∞ ω
supremum norm: sup |f|
(n) Rn
periodized configuration
1
periodic empirical fields: |Vn| i∈Vn δθiω (n)
Λ a (sometimes finite) subset of Zd Vn
the box {i ∈ Zd : −n < i1, . . . , id < n}
νΛ
the restriction of ν ∈ M1(X Zd)
to X Λ
νn
the restriction of ν ∈ M1(X Zd)
to X Vn
HΛ(ν | λ) entropy of νΛ relative to λΛ
Hn(ν | λ) entropy of νn relative to λn
h(ν | λ) specific entropy of ν relative to λ

Notation index 195
B space of absolutely summable shift-invariant
interaction potentials
Hfree Λ
H
Hamiltonian in volume Λ with free boundary conditions
τΛc Λ Hamiltonian in volume Λ with boundary condition τΛc πΛ(τ, ·) specification with boundary condition τΛc πτ Λ (·)
Π
specification with boundary condition τΛc specification
G Π set of Gibbs measures consistent with specification Π
H0 Λ
µ
free Ising Hamiltonian
ω Λ,β,h,J
µ
Ising specification with boundary ω, parameters (β, h, J)
0 Λ,β,h,J
µ
Ising specification with free boundary condition
+
Λ,β,h,J
µ
Ising specification with boundary ω ≡ 1
−
Λ,β,h,J Ising specification with boundary ω ≡ −1
x ∧ y min(x, y)
x ∨ y max(x, y)
ω ≤ σ partial order on configurations: ωi ≤ σi ∀i ∈ Zd µ ≤ ν ν stochastically dominates µ
P (Φ) infinite volume pressure corresponding to potential Φ
h(ν | Φ) specific entropy relative to potential Φ
κ upper rate function
κ lower rate function
¯p upper pressure
J convex conjugate of ¯p
ex(K) extreme points of K
αq push forward of measure α by kernel q

Formula
Stirling’s,
Theorems, principles, and models index
Inequality
arithmetic-geometric,
Chebyshev’s,
Doob’s,
Fenchel-Young,
GHS,
Griffiths’,
Grifiths-Hurst-Sherman,
Hölder’s,
Holley’s,
Jensen’s,
Lemma
Borel-Cantelli,
Fatou’s,
Feteke’s,
Strassen’s,
Varadhan’s,
Model
Curie-Weiss,
Fortuin-Kasteleyn,
Ising,
plane rotator,
plane rotor,
XY,
Principle
contraction,
DLR variational,
Dobrushin-Lanford-Ruelle variational,
Gibbs conditioning,
large deviation,
maximum entropy,
Maxwell’s,
push-forward,
Theorem
analytic implicit function,
backwards-martingale convergence,
Baxter-Jain,
Bryc’s,
Choquet’s,
Cramér’s,
de Finetti’s,
dominated convergence,
ergodic,
ergodic decomposition,
Fenchel-Morreau,
Hahn-Banach separation,
Kolmogorov’s extension,
Markov chain convergence,
minimax,
monotone class,
monotone convergence,
portmanteau,
Prohorov’s,
Riesz representation,
Sanov’s,
Tychonov’s,
Ulam’s,
Varadhan’s,

Aizenman, 113
Bartle, 6
Baxter and Jain, 146
Beijeren, see van Beijeren
Billingsley, 169
Boltzmann, 69
Bricmont, Fontaine, and Landau, 96
Dembo and Zeitouni, 23, 24, 35, 51, 52, 65
Deuschel and Stroock, 173
Dinwoodie, 24
Dobrushin, 93, 94, 113, 116
Dudley, 143, 168
Durrett, 4, 11, 12, 21–23, 26, 49, 51, 58, 59,
64, 67, 76, 91, 97, 113, 123, 135, 137,
175, 184
Dynkin, 173
Feller, 4
Folland, 109, 143, 172
Fontaine, see Bricmont, Fontaine, and
Landau
Fröhlich and Pfister, 96
Fröhlich and Spencer, 96
Fritzsche and Grauert, 93
Georgii, 98, 113, 171
Gibbs, 69
Grauert, see Fritzsche
Griffiths, 116
Higuchi, 113
Jain, see Baxter and Jain
König, 178
Kassay, 178
Khoshnevisan, 4, 12, 22, 23, 26, 49, 51, 58,
59, 64, 67, 76, 113, 123, 137, 175, 178
Landau, see Bricmont, Fontaine, and
Landau
Lanford and Ruelle, 93
Author index
Maxwell, 69
Messager, Miracle-Sole, and Pfister, 96
Miracle-Sole, see Messager, Miracle-Sole,
and Pfister
Parthasarathy, 143, 168, 170
Pfister, see Fröhlich and Pfister, see
Messager, Miracle-Sole, and Pfister
Phelps, 97
Rockafellar, 44
Rudin, 42, 43, 81, 142, 149, 150, 168
Ruelle, see Lanford and Ruelle
Schrödinger, 6
Seppäläinen and Yukich, 36
Spencer, see Fröhlich and Spencer
Stroock, see Deuschel and Stroock
van Beijeren, 113
Yukich, see Seppäläinen and Yukich
Zeitouni, see Dembo and Zeitouni

additive, 7
affine, 44, 75, 103
minorant, 41, 43, 47
analytic function, 93, 161
analytic implicit function theorem, 93
arithmetic-geometric inequality, 159
atom, 37
backwards-martingale convergence
theorem, 97, 113, 175
Baxter-Jain theorem, 146
Berry-Esseen theorem, 135, 136
biconjugate, see convex
Borel-Cantelli lemma, 12, 27
boundary condition, 89, 99, 101, 112, 112,
116, 119, 125
free, 89, 102
Bryc’s theorem, 31, 35, 35, 56
canonical ensemble, 69
Chabyshev’s inequality, 52
Chebyshev’s inequality, 11, 21, 137
chemical potential, 94
Choquet’s theorem, 97
circuit, 116–118
compact
set, 16, 17, 19, 20, 25, 27, 30, 36, 43, 49,
60, 61, 63, 66, 68, 75, 76, 81, 82,
103, 109, 113, 145, 146, 149, 150,
155, 158, 159, 161, 169, 170
topological space, see topological space
compactification, 145, 159, 160
concave
function, 25, 26, 35, 75, 126, 127, 155
uniformly Jensen-concave-covex-like, 178
conditional probability, 62, 69, 78, 90, 91,
142, 144, 171–173
conjugate, see convex
contour, see circuit, 117
contraction principle, 29, 29, 30, 31, 50, 57,
65, 161
Bernoulli, 30, 39
convex
biconjugate, 43, 46, 142
General index
conjugate, 43, 49, 51, 58, 60, 65, 80, 143,
145, 156
duality, 35, 41, 42, 49, 58, 60, 145, 155,
167, 168
function, 22, 25, 26, 36, 41, 43, 43, 44,
46–51, 54, 56, 64, 75, 77, 125, 126,
142, 143, 145–149, 154, 156, 157
hull, 23, 41, 46, 46, 149, 150
minorant, 46, 47
rate function, see rate function
set, 25, 41, 43, 44, 49, 52–54, 68, 72, 76,
109, 142, 145, 146, 149, 150, 155, 171
strictly, 58, 59, 66, 68
uniformly Jensen-concave-covex-like, 178
correlation function, 119–121
coupling, 183
covex
function, 155
Cramér’s theorem, 20, 21, 23, 24, 27, 37,
39, 56, 65, 66, 71, 78, 136
R, 23, 133
R d , 24, 51, 51, 54, 56
Polish, 65
Polish space, 65
refinement, 28, 133, 133, 134
Curie point, 38
Curie-Weiss model, 9, 37, 37, 38, 88, 93,
94, 102, 111, 112
de Finetti’s theorem, 56, 174, 175
diagonal trick, 76, 126
DLR
equations, 93
variational principle, 9, 105
Dobrushin-Lanford-Ruelle
equations, 93
variational principle, 105
dominated convergence theorem, 26, 59,
113, 123, 127, 137
Doob’s inequality, 12
droplet, 115, 116
duality, see convex duality
Edgeworth expansion, 136
empirical

202 General index
average, 99
field, 73, 78, 101, 102, 104
periodic, 73
periodized, 104
fields
periodized, 78
mean, see sample mean, see sample mean
measure, 30, 31, 57, 62, 65, 71, 153, 156,
175
Bernoulli, 57
energy, 6–8, 37, 38, 65–67, 69, 87, 89, 99,
102, 115, 118
free, 36, 67, 67, 88, 93, 95, 101, 102
enthalpy, 67
entropy, 5, 8, 66–69, 73, 88, 102, 141
conditional, 62, 78, 108, 142
information-theoretic, 5, 57
maximum, 66, 67, see maximum entropy
principle, 68
minimum, 66, 68
relative, 4, 5, 57, 58, 59, 60, 62–65, 67,
68, 73, 74, 77, 78, 88, 108, 153, 154,
163
Bernoulli, 31, 58
specific, 73, 74, 75, 77, 101, 102, 173
thermodynamic, 5, 6, 7, 8, 57, 67, 68
epigraph, 15, 16, 41, 44, 45–47
equilibrium, 87, 90, 91, 94, 99
equivalence of ensembles, 69, 69, 70, 88
ergodic
measure, 72, 77, 96, 97, 170, 170,
171–173
ergodic decomposition theorem, 171, 173,
174
ergodic theorem, 72, 73, 76, 79, 83, 97, 170,
170, 171
exchangeable, 56, 173, 174, 174, 175
exponentially tight, 17, 19, 19, 20, 25, 35,
36, 54, 62, 63, 79, 81, 82, 145, 158, 160
external magnetic field, see magnetic field
extreme point, 72, 72, 76, 97, 98, 113, 149,
150, 171, 171
Fatou’s lemma, 51
Fekete’s lemma, 53
multiindex, 74
Feller-continuous, 92, 92
Fenchel-Morreau theorem, 44
Fenchel-Young inequality, 44, 45, 48
ferromagnet, 37, 39, 101
finite range, see range
Fortuin-Kasteleyn model, 129
free energy, see energy, 94
GHS inequality, see
Griffiths-Hurst-Sherman inequality
Gibbs
conditioning principle, 67, 136
measure, 35, 37, 66, 67, 69, 83, 87, 88,
89, 90, 92–98, 100, 102–105, 109,
112, 113, 115, 116, 119–121, 123,
126, 173
specification, see specification
Griffiths’ inequality, 124, 184, 185
mybf, 185
Griffiths-Hurst-Sherman inequality, 186,
187
Grifiths-Hurst-Sherman inequality, 126,
186
Hölder’s inequality, 26, 49, 80, 126, 142
Hahn-Banach separation theorem, 42, 45,
47, 48, 142, 146, 150, 151, 180
Hamiltonian, 37, 38, 65, 87–89, 99, 100,
106, 112, 118, 122
free, 112
Hausdorff, see topological space
Helmholtz free energy, 88
Hewitt-Savage 0-1 law, 175
Holley’s inequality, 122, 183, 183
inclusion-exclusion, 119
Inequlaity
Jensen, 67
information, 6, 8
interaction, see ptential88
interaction potential, see potential
intermittency, 34
inverse temperature, 66, see temperature,
93, 94, 99, 111
critical, 113
Ising model, 37, 38, 94, 95, 106, 110, 111,
115, 125, 126, 129, 186
one-dimensional, 113, 115
two-dimensional, 116
Jensen’s inequality, 22, 58, 64, 83, 142, 145,
154, 155, 159
Kirkwood-Salzburg
equation, 120
kernel, 119
Kolmogorov’s 0-1 law, 77, 172
Kolmogorov’s extension theorem, 76, 123
Krein-Milman theorem, 149
Lagrange multiplier, 6, 7
large deviation, 3–5, 8, 11, 12, 16, 17, 20,
21, 27, 41, 57, 71, 73, 83, 84, 99, 100,
103, 105, 134, 136, 145, 151, 153
Bernoulli, 4
informal, 11
level 1, 71
level 2, 71

General index 203
level 3, 71
lower bound, 17, 19, 20, 26, 29, 32
Markov chain, 140, 144
position level, 71
process level, 71, 82
upper bound, 17, 19, 20, 22, 24, 25, 27,
29, 32, 33, 40, 49, 82, 162
large deviation principle (LDP), 12, 14, 16,
16, 19–24, 26, 27, 29–32, 34–37, 39, 40,
50–52, 54, 57, 62, 65, 73, 76, 78, 82,
89, 101, 102, 104, 105, 146, 148–151,
155, 156, 158–163, 194
Bernoulli, 13
exchangeable process, 56
Normal, 20
uniform, 104
weak, 19, 19, 20, 24, 25, 36, 54
lattice gas, 101
LDP, see large deviation principle
level 1, see large deviation
level 2, see large deviation
level 3, see large deviation, see process level
Lipschitz function, 35
liquid gas phase transition, 113, see phase
transition
local function, 72
locally convex space, see topological space
lower semicontinuous, 15, 15, 16, 22, 27,
30, 32, 34, 36, 41, 43, 44, 46–51, 56,
60, 61, 66, 68, 75, 77, 103, 109, 142,
143, 146–150, 173
minorant, 15, 41, 46, 47
not, 30
regularization, 15, 16, 29, 41, 44, 47
macroscopic, 87
macrostate, 96–98
magnetic field, 112
external, 37, 39, 88, 89, 93, 111, 112
magnetization, 39
spontaneous, 37, 39, 111
Markov chain, 62, 90, 115, 141, 183, 184
kernel, 144
Markov chain convergence theorem, 62, 184
Markov field, 115
martingale, 12
maximum entropy principle, 67, 67, 68, 69,
84
Maxwell’s principle, 70
mean-field approximation, 35, 37, 38
measure
a priori, 88, 89
reference, 88
metric, see topological space
metrize, 17, 168, 169
metrized, 168
microcanonical ensemble, 69
microscopic, 87, 99
Milman’s theorem, 150
minimal spanning tree, 36
minimax theorem
R d , 25
minimizer, 34, 61, 66–69, 84, 105
moderate deviation, 27, 28, 136
moment generating function, 21, 24, 24, 51
monotone class, 172, 178
monotone class theorem, 178
monotone convergence theorem, 59
multidimensional ergodic theorem, see
ergodic theorem
normalization, 13, 16, 19, 20, 22, 31–33, 35,
145, 146, 155, 156, 161
constant, 37, 88, 90
observable, 96
macroscopic, 96, 96
microscopic, 96, 96, 99
pair potential, 89
partition function, 37, 66, 88, 90, 101, 112
Peierls argument, 116
Peierls arugment, 115
percolation, 128
periodized configuration, 73
permutation, 174
phase diagram, 39, 94, 95
phase transition, 88, 92–94, 94, 95, 96, 105,
110–113, 115, 116, 119, 121, 123–125,
128
liquid gas, 113
liquid-gas, 94, 95
plane rotator model, 96
plane rotor model, see plane rotator
Polish, see topological space
portmanteau theorem, 63, 72, 81, 159, 167,
168, 169
position level, see large deviation
potential, 88, 89, 93, 111
interaction, 88, 88, 89, 89, 93, 99
nearest-neighbor, 89
one body, 89
pair, 89
self, 89
two-body, 89
pressure, 50, 50, 78–80, 93–95, 101, 105,
124, 125, 146, 148, 156, 158
upper, 145
process level, 71, see large deviation
Prohorov
metric, 169, 170
Prohorov’s theorem, 61
Prohorov’s theorem, 63, 81, 169, 170
proper, 43, 44–46, 48, 51, 146

204 General index
proper function, 44
push forward, 144
push-forward principle, see contraction
principle
Radon-Nikodym derivative, 64, 83, 135
random cluster, see Fortuin-Kasteleyn
random field, 71, 72
range, 89
finite, 89
rate function, 4, 5, 8, 13–16, 16, 17, 20, 22,
23, 29–35, 37, 39, 40, 49–51, 54, 56, 60,
65, 66, 73, 104, 105, 109, 146, 148, 150,
151, 155, 156, 158, 159, 161–163
Bernoulli, 31
convex, 35, 40, 52, 62, 65
empirical measure
Bernoulli, 57
good, 17
lower, 20, 146, 155
lower semicontinuous, 16, 19, 27, 30, 146
not convex, 35, 40, 50, 50
not lower semicontinuous, 30
not tight, 30
tight, 16, 17, 20, 24, 27, 29, 30, 35, 52,
54, 62, 65, 73, 78, 104, 143, 154
unique, 17
upper, 20, 146, 155
zero, 22, 27, 35, 50, 76
regular
point, 148, 148, 149, 150
probability measure, 61
sequence, 155
topological space, see topological space
regularization, see lower semicontinuous
relative entropy, see entropy
Riesz representation, 143
sample mean, 14, 20–24, 31, 39, 51, 56, 57,
65, 71, 94
Sanov’s theorem, 62, 62, 65, 67, 71, 73, 76,
78, 81, 151, 153
self-potential, see potential
separable, see topological space
shift group, 72
shift-invariant, 75
event, 72, 77, 170, 170
function, 96, 101
Gibbs measuer, 123
Gibbs measure, 102, 105, 113, 123
measure, 72, 76, 102, 170–173
observable, see function
potential, 88, 89, 93, 97, 99, 105, 109
set, see event
specific entropy, see entropy
specification, 90, 91, 91, 92, 93, 96, 97, 112,
115, 123
Gibbs, 92–94, 103
spin, 37, 38, 72, 87–89, 93, 96, 109, 111,
112, 115–118, 122, 176, 183–185
configuration, 37
flip, 112, 116–118, 122
spin glass, 36
spontaneous magnetization, 37, see
magnetization, 39
statistical mechanics, 5, 8, 35, 37
Stirling’s formula, 31
Stirling’s formula, 4, 6, 14
stochastic domination, 121
stochastic kernel, 90, 90, 91, 97, 98, 141,
144, 163, 173–175
Strassen’s lemma, 121, 175
strictly convex, see convex
subadditivity, 52, 55
Fekete’s lemma, 53, 74
subdifferential, 48, 50
sublevel set, 60, 61, 66, 68, 75, 103
tail σ-algebra, 172
tangent hyperplane, 48
temperature, 6, 39, 88
critical, 38
high, 38
inverse, 37, 66, 88
low, 38
tight
family of measures, 19, 30, 61, 63, 81,
159, 169, 169
rate function, see rate function
tolopogy
weak, 159
topological space, 169
Banach, 42, 89
compact, 92, 109, 142, 143, 145–147, 149,
150, 156, 158, 160, 161, 168, 179–181
Hausdorff, 12, 17, 19, 29, 30, 42, 145,
155, 167, 179
locally convex, 42, 42, 45
metric, 15–17, 26, 35, 58, 60, 75, 143,
145, 160, 161, 167–170, 172, 177
Polish, 60, 60, 61, 62, 65, 71, 72, 77, 87,
91, 141, 143, 144, 153, 158, 160,
170–172, 174
regular, 17
separable, 168–170, 172
totally bounded, 168
vector space, 42, 168
topology
Euclidean, 42
weak, 41, 42, 58, 60–63, 68, 72, 75, 82,
89, 109, 142, 145, 153, 155, 159,
167–169
weak ∗ , 42, 143

General index 205
totally bounded, 159, 168, see topological
space
metric, 143
transfer matrix, 114
transition matrix, 115
traveling salesman, 36
Tychonov’s theorem, 81
Ulam’s theorem, 61, 63, 170
uniform integrability, 61, 61
upper semicontinuous, 32, 34, 147, 148,
179, 180
Varadhan’s lemma, see Varadhan’s theorem
Varadhan’s theorem, 31, 31, 34, 35, 37, 50,
51, 93, 101, 104, 162
weak convergence, 13, 30, 35, 37, 38, 40,
68–70, 72, 73, 75–77, 81, 84, 92, 113,
116, 123, 159, 161, 162, 167, 167, 168,
169
weak large deviation, 19
weak topology, see topology
XY model, see plane rotator