Foundations of Data Science

6.5 Online learning and the Perceptron algorithm
So far we have been considering what is often called the batch learning scenario. You are
given a “batch” of data—the training sample S—and your goal is to use it to produce
a hypothesis h that will have low error on new data, under the assumption that both S
and the new data are sampled from some fixed distribution D. We now switch to the
more challenging online learning scenario where we remove the assumption that data is
sampled from a fixed probability distribution, or from any probabilistic process at all.
Specifically, the online learning scenario proceeds as follows. At each time t = 1, 2, . . .:
1. The algorithm is presented with an arbitrary example x t ∈ X and is asked to make
a prediction l t of its label.
2. The algorithm is told the true label of the example c ∗ (x t ) and is charged for a
mistake if c ∗ (x t ) ≠ l t .
The goal of the learning algorithm is to make as few mistakes as possible in total. For
example, consider an email classifier that when a new email message arrives must classify
it as “important” or “it can wait”. The user then looks at the email and informs the
algorithm if it was incorrect. We might not want to model email messages as independent
random objects from a fixed probability distribution, because they often are replies to
previous emails and build on each other. Thus, the online learning model would be more
appropriate than the batch model for this setting.
Intuitively, the online learning model is harder than the batch model because we have
removed the requirement that our data consists of independent draws from a fixed probability
distribution. Indeed, we will see shortly that any algorithm with good performance
in the online model can be converted to an algorithm with good performance in the batch
model. Nonetheless, the online model can sometimes be a cleaner model for design and
analysis of algorithms.
6.5.1 An example: learning disjunctions
As a simple example, let’s revisit the problem of learning disjunctions in the online model.
We can solve this problem by starting with a hypothesis h = x 1 ∨ x 2 ∨ . . . ∨ x d and using
it for prediction. We will maintain the invariant that every variable in the target disjunction
is also in our hypothesis, which is clearly true at the start. This ensures that the
only mistakes possible are on examples x for which h(x) is positive but c ∗ (x) is negative.
When such a mistake occurs, we simply remove from h any variable set to 1 in x. Since
such variables cannot be in the target function (since x was negative), we maintain our
invariant and remove at least one variable from h. This implies that the algorithm makes
at most d mistakes total on any series of examples consistent with a disjunction.
198