A Course on Large Deviations with an Introduction to Gibbs Measures.

4 1. Introduction
Example 1.1. Let us consider coin tosses. Let {Xn} be an i.i.d. sequence of
Bernoulli random variables with success probability p (i.e. each Xn = 1 with
probability p and 0 otherwise). Denote the partial sum by Sn = X1+· · ·+Xn.
The law of large numbers ([15] or page 73 of [26]) says that Sn/n converges
to p, almost surely. But at any given n there is a chance of p n that we get all
heads (Sn = n) and also a chance of (1 − p) n that we get all tails (Sn = 0).
In fact, for any s ∈ (0, 1) there is always a chance that one gets a fraction
of heads close to s. Let us compute this probability.
Let us write [x] for the integral part of x ∈ R, i.e. the largest integer
smaller or equal to x. Write
P {Sn = [ns]} =
n!
[ns]!(n − [ns])! p[ns] (1 − p) n−[ns]
∼ nn p [ns] (1 − p) n−[ns]
[ns] [ns] (n − [ns]) n−[ns]
n
2π[ns](n − [ns]) ,
where we have used Stirling’s formula n! ∼ e−nnn√2πn; see, for example,
page 21 of Khoshnevisan’s textbook [26] or page 52 of Feller’s Vol. I [17].
(We say that an ∼ bn, or an is equivalent to bn, when an/bn → 1.) Let us
abbreviate
n
βn =
2π[ns](n − [ns]) ,
γn = (ns)ns (n − ns) n−ns
[ns] [ns] (n − [ns]) n−[ns]
p [ns] (1 − p) n−[ns]
pns (1 − p) n−ns .
Then, P {Sn = [ns]} is equivalent to
βnγn exp{n log n+ns log ns+n(1−s) log n(1−s)−ns log p−n(1−s) log(1−p)}.
∗ Exercise 1.2. Show that there exists a constant C such that 1
C √ n ≤ βn ≤
C and 1
Cn ≤ γn ≤ Cn for large enough n.
(1.1)
One then has
lim
n→∞
1
n log P {Sn = [ns]} = −Ip(s), with
Ip(s) = s log s
1 − s
+ (1 − s) log
p 1 − p .
This function Ip is continuous on (0, 1) and its limits at 0 and 1 are exactly
what we predicted earlier: Ip(1) = log 1
p and Ip(0) = log 1
1−p . For s ∈ [0, 1]
it is natural to set Ip(s) = ∞. Figure 1.1 shows what this function looks
like.
The function Ip in (1.1) is called a large deviation rate function. Ip(s) is
also called the entropy of the coin yielding heads with probability s relative