Algorithm Design

720
Chapter 13 Randomized Algorithms
declaring this to have the value c~ if the sum diverges. Thus, for example,
if X takes each of the values in {1, 2, n} with probability l/n, then
E IX] = 1(!/n) + 2(l/n) +-" + n(lln) --- (n-~l)/n --- (n + 1)/2.
Example: Waiting for a First Success
Here’s a more useful example, in which we see how an appropriate random
variable lets us talk about something like the "running time" of a simple
random process. Suppose we have a coin that comes up heads with probability
p > 0, and ~cails with probability 1- p. Different flips of the coin have
independent outcomes. If we flip the coin until we first get a heads, what’s
the expected number of flips we will perform? To answer this, we let X denote
the random variable equal to the number of flips performed. For j > 0, we
h ~,_ r X ;~ r 1 p)i-lp, in order for the process to take exactly j steps,
the first j - 1 flips must come up ta±ls, and the j~" must come up heads.
Now, applying the definition, we have
E[X] =~_~J’Pr[X~J]=~__, j(1-p)]-lp- 1 p 7= 1
~=o 7=I
= ~_2_. 0- P___2 = !.
1 -- p p2 p
Thus we get the following intuitively sensible result.
(13.7) If we repeatedly perform independent trials of an experimer& each of
which succeeds with probability p > 0, then the expected number of trials we
need to perform until the first success is 1/p.
Linearity of Expectation ....
In Sections 13.1 and 13.2, we broke events down into unions of mudh simpler
events, and worked with the probabilities of these simpler events. This is a
powerful technique when working with random variables as well, and it is
based on the principle of linearity of expectation.
(13.8) Linearity of Expectation. Given two random variables X and Y defined
over the same probability space, we can define X + Y to be the random variable
equal to X(~o) + Y(co) on a sample point o~. For any X and Y, we have
~ [X + ~] = ~ [X] + ~ [V] . .......
We omit the proof, which is not difficult. Much of the power of (13.8)
comes from the fact that it applies to the sum of any random variables; no
restrictive assumptions are needed. As a result, if we need to compute the
13.3 Random Variables and Their Expectations
expectation of a complicated random variable X, we can first write it as a
sum of simpler random variables X = X 1 + X 2 +. ¯ ¯ + Xn, compute each E [Xi],
and then determine E [X] = y~ E [Xi]. We now look at some examples of this
principle in action.
Example: Guessing Cards
Memoryless Guessing To amaze your friends, you have them shuffle a deck
of 52 cards and then turn over one card at a time. Before each card is turned
over, you predict its identity. Unfortunately, you don’t have any particular
psychic abilities--and you’re not so good at remembering what’s been turned
over already--so your strategy is simply to guess a card uniformly at random
from the full deck each time. On how many predictions do you expect to be
correct?
Let’s work this out for the more general setting in which the deck has n
distinct cards, using X to denote the random variable equal to the number of
correct predictions. A surprisingly effortless way to compute X is to define the
random variable Xi, for i = 1, 2 ..... n, to be equal to 1 if the i th prediction is
correct, and 0 otherwise. Notice that X = X~ + X 2 + ¯ ¯ ¯ + X n, and
[x,] = o . *’r IX, = O] + 1. er IX, = q = er IX, = 1] = i.
It’s worth pausing to note a useful fact that is implicitly demonstrated by the
above calculation: If Z is any random variable that only takes the values 0 or
1, then E [Z] = Pr [Z = !].
1 for each i, we have
Since E [Xi] = -~
E [X] = 2 E [X,] = n (-~ ) = I.
f=l
Thus we have shown the following.
(13.9) The expected number of correct predictions under the memoryIess
guessing strategy is 1, independent of n.
Trying to compute E [X] directly from the definition ~0J" Pr [X =j]
would be much more painful, since it would involve working out a much more
elaborate summation. A significant amount of complexity is hidden away in
the seemingly innocuous statement of (13.8).
Guessing with Memory Now let’s consider a second scenario. Your psychic
abilities have not developed any further since last time, but you have become
very good at remembering which cards have already been turned over. Thus,
when you predict the next card now, you only guess uniformly from among
721