Algorithm Design

758
Chapter 13 Randomized Algorithms
This is the probability of a miss on the request to s. Summing over all requests
to unmarked items, we have
i=1
Thus the total expected number of misses incurred by the Randomized
Marking Algorithm is
r
i=1 i=1
Combining (13.39) and (13.40), we immediately get the following perfor-
mance guarantee.
(13.41) The expected nwraber of misses incurred by the Randomized Mafking
Algorithm is at most 2H(k) ¯ f(a) = O(log k)- f(a).
13.9 Chernoff Bounds
In Section 13.3, we defined the expectation of a random variable formally and
have worked with this definition and its consequences ever since. Intuitively,
we have a sense that the value of a random variable ought to be "near" its
expectation with reasonably high probability, but we have not yet explored
the extent to which this is true. We now turn to some results that allow us to
reach conclusions like this, and see a sampling of the applications that follow.
We say that two random variables X and Y are independent if, for any
values i and j, the events Pr IX = i] and Pr [Y =j] are independent. This
definition extends naturally to larger sets of random variables. Now consider
a random variable X that is a sum of several independent 0-1-valued random
variables: X = X! + X~. +. ¯ ¯ + Xn, where Xi takes the value I with probability
otherwise. By linearity of expectation, we have E [X] =
pi, and the value 0 the independence of the random variables X1, X2 .....Xn
~=~ p~. Intuitively,
suggests that their fluctuations are likely to "cancel out;’ and so their sum
X will have a value close to its expectation with high probability. This is in
fact true, and we state two concrete versions of this result: one bounding the
probability that X deviates above E[X], the other bounding the probability that
X deviates below E[X]. We call these results Chemoff bounds, after one of the
probabilists who first established bounds of this form.
r
13.9 Chernoff Bounds
Proof. To bound the probability that X exceeds (1 + 8)/z, we go through a
sequence of simple transformations. First note that, for any t > 0, we have
Pr [X > (1 + 8)/z] = Pr [e tx > et(l+~)u], as the function f(x) = e tx is monotone
in x. We will use this observation with a t that we’ll select later.
Next we use some simple properties of the expectation. For a random
variable Y, we have y Pr [Y > ~] _< E [Y], by the definition of the expectation.
This allows us to bound the probability that Y exceeds y in terms of E [Y].
Combining these two ideas, we get the following inequalities.
Next we need to bound the expectation E [etX]. Writing X as X = Ei Xi, the
expectation is E [e tx] = E [et ~x~] = E [r-[i etX;] ¯ For independent variables Y
and Z, the expectation of the product YZ is E [YZ] = E [Y] E [Z]. The variables
Xi are independent, so we get E [F[ie tx~] = r-liE [etX~].
Now, e a~ is e ~ with probability Pi and e ° = 1 otherwise, so its expectation
can be bounded as
E [e a*] =pier+ (1 -Pi) = 1 +pi(e t - 1) < e p6e’-l),
where the last inequality follows from the fact that 1 + a _< e ~ for any ~ >_ 0.
Combining the inequalities, we get the following bound.
--< e-t(l+~)tz H ePz(e~-l)