Algorithm Design

772
Chapter 13 Randomized Algorithms
Pr [£ rn 5:1 _ Pr [£],
Pr
and hence Pr [E N 9:] = Pr (El. Pr [5:], from which the other equality holds as
well.
It turns out to be a little cleaner to adopt this equivalent formulation as
our working definition of independence. Formally, we’ll say that events £ and
5: are independent if Pr[g r3 9:] = Pr [£]. Pr [9:].
This product formulation leads to the following natural generalization. We
say that a collection of events gx, E2 ..... £n is independent if, for every set of
indices I _ {1, 2 ..... n}, we have
It’s important to notice the following: To check if a large set of events
is independent, it’s not enough to check whether every pair of them is
independent. For example, suppose we flip three independent fair coins:, If £i.
denotes the event that the i th coin comes up heads, then the events gx, gz, E3
are independent and each has probability 1/2. Now let A denote the event that
coins 1 and 2 have the same value; let B denote the event that coins 2 and 3 have
the same value; and let C denote the event that coins 1 and 3 have different
values. It’s easy to check that each of these events has probability 1/2, and the
intersection of any two has probability 1/4. Thus every pair drawn from A, B, C
is independent. But the set of all three events A, B, C is not independent, since
Pr [A~BOC]=O.
i~I
The Union Bound
Suopose we are given a set of events El, ~2 .....
En, and we are interested
in the probability that any of them happens; that is, we are interested in the
probaNlity Pr [t-J~___lEi]. If the events are all pairwise disjoint from one another,
then the probability mass of their union is comprised simply of the separate
contributions from each event. In other words, we have the following fact.
pai~.
ti=l ._I i=1
In general, a set of events E~, ~2 ..... ~n may overlap in complex ways. In
this case, the equality in 03.49) no longer holds; due to the overlaps among
E1
E2 E 3
13.12 Background: Some Basic Probability Definitions
Figure 13.5 The Union Bound: The probability of a union is maximized when the events
have no overlap.
events, the probability mass of a point that is counted once on the left-hand
side will be counted one or more times on the right-hand side. (See Figure 13.5.)
This means that for a general set of events, the equality in (13.49) is relaxed to
an inequality; and this is the content of the Union Bound. We have stated the
Union Bound as (13.2), but we state it here again for comparison with (13.49).
(13.50} (The Union Bound) Given events E~, ~2 ..... E n, we have
Pr i < Pr
i=1 i=!
..... .......... .... ~ : ....... :~ ~
Given its innocuous appearance, the Union Bound is a surprisingly powerful
tool in the analysis of randomized algorithms. It draws its power mainly
from the following ubiquitous style of analyzing randomized algorithms. Given
a randomized algorithm designed to produce a correct result with high probability,
we first tabulate a set of "bad events" ~1, ~2 ..... ~n with the following
property: if none of these bad events occurs, then the algorithm will indeed
produce the correct answer. In other words, if 5 ~ denotes the event that the
algorithm fails, then we have
Pr[Sq