Algorithm Design

740
Chapter 13 Randomized Algorithms
This now completes our random implementation of dictionaries. We define
the family of hash functions to be 9£--= {ha:a ~ A]- To execute MakeDictionary,
we choose a random hash function from J~; in other words, we
choose a random vector from A (by choosing each coordinate uniformly at
random), and form the function ha. Note that in order to define A, we need
to find a prime number p >_ n. There are methods for generating prime numbers
quickly, which we will not go into here. (In practice, this can also be
accomplished using a table of known prime numbers, even for relatively large
n.) We then use this as the hash function with which to implement Insert,
Delete, and Lookup. The family 9~ = {ha : a ~ A} satisfies a formal version of
the second property we were seeking: It has a compact representation, since
by simply choosing and remembering a random a ~ A, we can compute ha(u)
for all elements u ~ U. Thus, to show that ~ leads to an efficient, hashingbased
implementation of dictionaries, we iust need to establish that ~ is a
universal family of hash functions.
~ Analyzing the Data Structure
If we are using a hash function ha from the class J~ that we’ve defined, then a
collision ha(X) : ha(Y) defines a linear equation modulo the prime number p. In
order to analyze such equations, it’s useful to have the following "cancellation
law."
(13.24) For any prime p and any integer z ~= 0 rood p, and any two integers
a, r, ifaz =/~z modp, then ~ =/~ modp.
ProoL Suppose az = ~z modp. Then, by rearranging terms, we get z(a -/~) =
0 mod p, and hence z(a - r) is divisible by p. But z ~ 0 rood p, so z is not
divisible by p. Since p is prime, it follows that a - fl must be divisible by p;
that is, a --- fl rood p as claimed. []
We now use this to prove the main result in our analysis.
(13.25) The class of linear fimctions ~K defined above is universal.
Proof. Let x = (Xl, x2 .... Xr) and y = (Y~, Yz ....Yr) be two distinct elements
of U. We need to show that the probability of ha(x) = ha(Y), for a randomly
chosen a ~ A, is at most 1/p.
Since x 5 & y, then there must be an index j such that xj ~ yj. We now
consider the following way of choosing the random vector a ~ A. We first
choose all the coordinates ai where i ~j. Then, finally, we choose coordinate
aj. We will show that regardless of how a!l the other coordinates ai were
13.7 Finding the Closest Pair of Points: A Randomized Approach 741
chosen, the probability of ha(x ) = ha(Y), taken over the final choice of aj, is
exactly 1/p. It will follow that the probability of ha(x) = ha(Y) over the random
choice of the full vector a must be 1/p as well.
This conclusion is intuitively clear: If the probability is 1/p regardless of
how we choose all other ai, then it is lip overall. There is also a direct proof
of this using conditional probabilities. Let £ be the event that ha(x) = ha(Y),
and let 5’ b be the event that all coordinates ai (for i ~j) receive a sequence of
values b. We will show, below, that Pr [~ I 5~b] = 1/p for all b. It then follows
that Pr [g] = ~b Pr" [~ I 9:b]- Pr [5:b] = (I/p) ~bPr [9:b] = !/p.
So, to conclude the proof, we assume that values have been chosen
arbitrarily for all other coordinates a i, and we consider the probability of
selecting aj so that ha(x ) = ha(Y ). By rearranging terms, we see that ha(x) =
ha(Y) if and only if
aj(Y] -- xj) = E ai(xi -- yi) modp.
Since the choices for all a i (i ~j) have been fixed, we can view the right-hand
side as some fixed quantity m. Also, let us define z =
Now it is enough to show that there is exactly one value 0 < a] < p that
satisfies a]z = m rood p; indeed, if this is the case, then there is a probability
of exactly lip of choosing this value for aj. So suppose there were two such
values, aj and a~. Then we would have ajz = a~z modp, and so by (13.24) we
would have a] = a~ rood p. But we assumed that a], a~ < p, and so in fact aj
and a~ would be the same. It follows that there is only one aj in this range that
satisfies ajz = m rood p.
Tracing back through the implications, this means that the probability of
choosing aj so that ha(x ) = ha(Y ) is l/p, however we set the other coordinates
a i in a; thus the probability that x and y collide is lip. Thus we have shown
that ~C is a universal class of hash functions, u
13.7 Finding the Closest Pair of Points:
A Randomized Approach
In Chapter 5, we used the divide-and-conquer technique to develop an
O(n log n) time algorithm for the problem of finding the closest pair of points in
the plane. Here we will show how to use randomization to develop a different
algorithm for this problem, using an underlying dictionary data structure. We
will show that this algorithm runs in O(n) expected time, plus O(n) expected
dictionary operations.
There are several related reasons why it is useful to express the running
time of our algorithm in this way, accounting for the dictionary operations