Foundations of Data Science

expected value of the sum will be zero where the expectation is over the choice of the ±1
value for the x s .
( m
)
∑
E x s f s = 0
s=1
Although the expected value of the sum is zero, its actual value is a random variable and
the expected value of the square of the sum is given by
( m
) 2 (
∑
m
)
)
∑
m∑
E x s f s = E x 2 sfs
2 + 2E x s x t f s f t = fs 2 ,
s=1
s=1
( ∑
s≠t
The last equality follows since E (x s x t ) = E(x s )E(x t ) = 0 for s ≠ t, using pairwise
independence of the random variables. Thus
( m
) 2
∑
a = x s f s
s=1
is an unbiased estimator of ∑ m
s=1 f s 2 in that it has the correct expectation. Note that at
this point we could use Markov’s inequality to state that Prob(a ≥ 3 ∑ m
s=1 f s 2 ) ≤ 1/3, but
we want to get a tighter guarantee. To do so, consider the second moment of a:
( m
) 4 ( )
∑
∑
E(a 2 ) = E x s f s = E
x s x t x u x v f s f t f u f v .
s=1
1≤s,t,u,v≤m
The last equality is by expansion. Assume that the random variables x s are 4-wise independent,
or equivalently that they are produced by a 4-wise independent hash function.
Then, since the x s are independent in the last sum, if any one of s, u, t, or v is distinct
from the others, then the expectation of the whole term is zero. Thus, we need to deal
only with terms of the form x 2 sx 2 t for t ≠ s and terms of the form x 4 s.
Each term in the above sum has four indices, s, t, u, v, and there are ( 4
2)
ways of
choosing two indices that have the same x value. Thus,
E(a 2 ) ≤
= 6
( 4
2)
E
m∑
( m
∑
m∑
s=1 t=s+1
m∑
s=1 t=s+1
f 2 s f 2 t +
s=1
) ( m
)
∑
x 2 sx 2 t fs 2 ft
2 + E x 4 sfs
4
m∑
s=1
f 4 s
( m
) 2
∑
≤ 3 fs 2 = 3E 2 (a).
s=1
Therefore, V ar(a) = E(a 2 ) − E 2 (a) ≤ 2E 2 (a).
s=1
245