Lectures on Elementary Probability

7.3. THE CHEBYSHEV INEQUALITY 47
Corollary 7.2 Let E 1 , E 2 , E 3 , . . . E n be events, each with probability p. Assume
that each pair E i , E j of events with i ≠ j is independent. Let f n be the proportion
of events that happen. Then the standard deviation of f n is
√
p(1 − p)
σ fn = √ . (7.10)
n
Proof: Let X 1 , X 2 , X 3 , . . . , X n be the corresponding Bernoulli random variables,
so that X i = 1 for outcomes in E i and X i = 0 for outcomes in the
complement of E i . Then the variance of X i is p(1 − p). The sample frequency
f n is just the sample mean of X 1 , . . . , X n . So its variance is obtained by dividing
by n.
It is very important to note that this result gives a bound for the variance
of the sample proportion that does not depend on the probability. In fact, it is
clear that
σ fn ≤ 1
2 √ n . (7.11)
This is an equality only when p = 1/2.
7.3 The Chebyshev inequality
Theorem 7.4 If Y is a random variable with mean µ and variance σ, then
P [|Y − µ| ≥ a] ≤ σ2
a 2 . (7.12)
Proof: Consider the random variable D that is equal to a 2 when |Y − µ| ≥ a
and is equal to 0 when |Y − µ| < a. Clearly D ≤ (Y − µ) 2 . It follows that
the expectation of the discrete random variable D is less than or equal to the
expectation of (Y − µ) 2 . This says that
a 2 P [|Y − µ| ≥ a] ≤ E[(Y − µ) 2 ] = σ 2 . (7.13)
Corollary 7.3 Let X 1 , X 2 , X 3 , . . . X n be random variables, each with mean µ
and standard deviation σ. Assume that each pair X i , X j of random variables
with i ≠ j is uncorrelated. Let
be their sample mean. Then
¯X n = X 1 + · · · + X n
n
(7.14)
P [| ¯X n − µ| ≥ a] ≤ σ2
na 2 . (7.15)
This gives a perhaps more intuitive way of thinking of the weak law of large
numbers. It says that if the sample size n is very large, then the probability that
the sample mean ¯X n is far from the population mean µ is moderately small.