Foundations of Data Science

is another situation in which we can derive a tighter bound than that given by the Chebyshev
inequality. We consider the case where the n independent variables are binomial but
similar results can be shown for independent random variables from any distribution that
has a finite variance.
Let x 1 , x 2 , . . . , x n be independent random variables where
x i =
{ 0 Prob 1 − p
1 Prob p
.
∑
Consider the sum s = n x i . Here the expected value of each x i is p and by linearity of
i=1
expectation, the expected value of the sum is m=np. Theorem 12.5 bounds the probability
that the sum s exceeds (1 + δ) m.
Theorem 12.3 For any δ > 0, Prob ( s > (1 + δ)m ) (
<
Proof: For any λ > 0, the function e λx is monotone. Thus,
) m
e δ
(1+δ) (1+δ)
Prob ( s > (1 + δ)m ) = Prob ( e λs > e λ(1+δ)m) .
e λx is nonnegative for all x, so we can apply Markov’s inequality to get
Since the x i are independent,
Prob ( e λs > e λ(1+δ)m) ≤ e −λ(1+δ)m E ( e λs) .
E ( e λs) = E
=
(
)
e λ ∑ n x i
i=1
= E
n∏ (
e λ p + 1 − p ) =
i=1
( n
∏
i=1
e λx i
Using the inequality 1 + x < e x with x = p(e λ − 1) yields
)
=
n∏
E ( e ) λx i
i=1
n∏ (
p(e λ − 1) + 1 ) .
i=1
Thus, for all λ > 0
E ( e λs) n∏
< e p(eλ−1) .
i=1
Prob ( s > (1 + δ)m ) ≤ Prob ( e λs > e λ(1+δ)m)
≤ e −λ(1+δ)m E ( e λs)
≤ e −λ(1+δ)m
n
∏
i=1
e p(eλ −1) .
398