Foundations of Data Science

Proof: Let x i = y i − p. Then, E(x i ) = 0 and E(x 2 i ) = E(y − p) 2 = p. For s ≥ 3,
|E(x s i )| = |E(y i − p) s |
= |p(1 − p) s + (1 − p)(0 − p) s |
= ∣ ∣p(1 − p) ( (1 − p) s−1 + (−p) s−1)∣ ∣
≤ p.
Apply Theorem 12.5 with a = cnp. Noting that a < √ 2 np, completes the proof.
Section (12.4.11) contains a different proof that uses a standard method based on
moment-generating functions and gives a better constant in the exponent.
Power Law Distributions
The power law distribution of order k where k is a positive integer is
f(x) = k − 1
x k for x ≥ 1.
If a random variable x has this distribution for k ≥ 4, then
µ = E(x) = k − 1
k − 2
and Var(x) =
k − 1
(k − 2) 2 (k − 3) .
Theorem 12.7 Suppose x 1 , x 2 , . . . , x n are i.i.d, each distributed according to the Power
Law of order k ≥ 4 (with n > 10k 2 ). Then, for x = x 1 + x 2 + · · · + x n , and any
ε ∈ (1/(2 √ nk), 1/k 2 ), we have
( ) (k−3)/2
4
Pr (|x − E(x)| ≥ εE(x)) ≤
.
ε 2 (k − 1)n
Proof: For integer s, the s th moment of x i − E(x i ), namely, E((x i − µ) s ), exists if and
only if s ≤ k − 2. For s ≤ k − 2,
E((x i − µ) s ) = (k − 1)
Using the substitution of variable z = µ/y
(y − µ) s
y k
∫ ∞
1
(y − µ) s
dy
y k
= y s−k (1 − z) s = zk−s
(1 − z)s
µ
k−s
As y goes from 1 to ∞, z goes from µ to 0, and dz = − µ dy. Thus
y 2
∫ ∞
E((x i − µ) s (y − µ) s
) =(k − 1)
dy
1 y k
= k − 1 ∫ 1
(1 − z) s z k−s−2 dz + k − 1 ∫ µ
(1 − z) s z k−s−2 dz.
µ k−s−1 µ k−s−1
0
404
1