Foundations of Data Science

Random variables x 1 , x 2 , . . . , x n are pairwise independent if for any a i and a j , i ≠ j,
Prob(x i = a i , x j = a j ) = Prob(x i = a i )Prob(x j = a j ). Mutual independence is much
stronger than requiring that the variables are pairwise independent. Consider the example
of 2-universal hash functions discussed in Chapter ??.
(
If (x, y) is a random vector and one normalizes it to a unit vector √ x √
y
x 2 +y 2 ,
x 2 +y 2 )
the coordinates are no longer independent since knowing the value of one coordinate
uniquely determines the value of the other.
12.4.2 Linearity of Expectation
An important concept is that of the expectation of a random variable. The expected
value, E(x), of a random variable x is E(x) = ∑ xp(x) in the discrete case and E(x) =
x
∞∫
xp(x)dx in the continuous case. The expectation of a sum of random variables is equal
−∞
to the sum of their expectations. The linearity of expectation follows directly from the
definition and does not require independence.
12.4.3 Union Bound
Let A 1 , A 2 , . . . , A n be events. The actual probability of the union of events is given
by Boole’s formula.
Prob(A 1 ∪ A 2 ∪ · · · A n ) =
n∑
i=1
Prob(A i ) − ∑ ij
Prob(A i ∧ A j ) + ∑ ijk
Prob(A i ∧ A j ∧ A k ) − · · ·
Often we only need an upper bound on the probability of the union and use
Prob(A 1 ∪ A 2 ∪ · · · A n ) ≤
This upper bound is called the union bound.
n∑
Prob(A i )
i=1
12.4.4 Indicator Variables
A useful tool is that of an indicator variable that takes on value 0 or 1 to indicate
whether some quantity is present or not. The indicator variable is useful in determining
the expected size of a subset. Given a random subset of the integers {1, 2, . . . , n}, the
expected size of the subset is the expected value of x 1 + x 2 + · · · + x n where x i is the
indicator variable that takes on value 1 if i is in the subset.
Example: Consider a random permutation of n integers. Define the indicator function
x i = 1 if the i th integer in the permutation is i. The expected number of fixed points is
given by
389