Theory of Statistics - George Mason University

840 Appendix B. Useful Inequalities in Probability
Corollary B.3.1.1 (Chebyshev’s inequality)
For ɛ > 0,
Pr(|X − E(X)| ≥ ɛ) ≤ 1
ɛ 2V(X)
Proof. In Markov’s inequality, let k = 2, and replace X by X − E(X).
Corollary B.3.1.2 (Chebyshev’s inequality (another form))
For f ∋ f(x) ≥ 0 and ɛ > 0,
(B.6)
Pr(f(X) ≥ ɛ) ≤ 1
E(f(X)) (B.7)
ɛ
Proof. Same as Markov’s inequality; start with E(f(X)).
Chebyshev’s inequality is often useful for ɛ = V(X). There are also
versions of Chebyshev’s inequality for specific families of distributions.
• 3σ rule for a unimodal random variable
If X is a random variable with a unimodal absolutely continuous distribution,
and σ = V(X), then
See Dharmadhikari and Joag-Dev (1988).
• Normal tail probability
If X ∼ N(µ, σ 2 ), then
See DasGupta (2000).
Pr(|X − E(X)| ≥ 3σ) ≤ 4
. (B.8)
81
Pr(|X − µ| ≥ kσ) ≤ 1
3k 2.
(B.9)
There are a number of inequalities that generalize Chebyshev’s inequality
to finite partial sums of a sequence of independent random variables
X1, X2, . . . over a common probability space such that for each, E(Xi) = 0
and E(X 2 i
) < ∞. (The common probability space means that E(·) has the
same meaning for each i, but the Xi do not necessarily have the same distribution.)
These inequalities are often called the The Bernstein inequalities,
but some of them have other names.
Theorem B.3.2 (the Hoeffding inequality)
Let X1, . . ., Xn be independent, E(Xi) = 0, and Pr(|Xi| ≤ c) = 1. Then for
any t > 0,
n
t
Pr Xi > t ≤ exp −
2 /2
n i=1 E(X2
. (B.10)
i ) + ct/3
i=1
The Hoeffding inequality is a special case of the Azuma inequality for martingales
(see Section 1.6).
Theory of Statistics c○2000–2013 James E. Gentle