Foundations of Data Science

estimate of m. To obtain a coin that comes down heads with probability 1/2 k , flip a fair
coin, one that comes down heads with probability 1 / 2 , k times and report heads if the fair
coin comes down heads in all k flips.
Given k, on average it will take 2 k ones before k is incremented. Thus, the expected
number of 1’s to produce the current value of k is 1 + 2 + 4 + · · · + 2 k−1 = 2 k − 1.
7.2.3 Counting Frequent Elements
The Majority and Frequent Algorithms
First consider the very simple problem of n people voting. There are m candidates,
{1, 2, . . . , m}. We want to determine if one candidate gets a majority vote and if so
who. Formally, we are given a stream of integers a 1 , a 2 , . . . , a n , each a i belonging to
{1, 2, . . . , m}, and want to determine whether there is some s ∈ {1, 2, . . . , m} which occurs
more than n/2 times and if so which s. It is easy to see that to solve the problem
exactly on read-once streaming data with a deterministic algorithm, requires Ω(min(n, m))
space. Suppose n is even and the last n/2 items are identical. Suppose also that after
reading the first n/2 items, there are two different sets of elements that result in the same
content of our memory. In that case, a mistake would occur if the second half of the
stream consists solely of an element that is in one set, but not in the other. If n/2 ≥ m
then there are at least 2 m − 1 possible subsets of the first n/2 elements. If n/2 ≤ m
then there are ∑ n/2
( m
)
i=1 i subsets. By the above argument, the number of bits of memory
must be at least the base 2 logarithm of the number of subsets, which is Ω(min(m, n)).
Surprisingly, we can bypass the above lower bound by slightly weakening our goal.
Again let’s require that if some element appears more than n/2 times, then we must
output it. But now, let us say that if no element appears more than n/2 times, then our
algorithm may output whatever it wants, rather than requiring that it output “no”. That
is, there may be “false positives”, but no “false negatives”.
Majority Algorithm
Store a 1 and initialize a counter to one. For each subsequent a i , if a i is the
same as the currently stored item, increment the counter by one. If it differs,
decrement the counter by one provided the counter is nonzero. If the counter
is zero, then store a i and set the counter to one.
To analyze the algorithm, it is convenient to view the decrement counter step as “eliminating”
two items, the new one and the one that caused the last increment in the counter.
It is easy to see that if there is a majority element s, it must be stored at the end. If
not, each occurrence of s was eliminated; but each such elimination also causes another
item to be eliminated. Thus for a majority item not to be stored at the end, more than
n items must have eliminated, a contradiction.
243