Machine Learning - DISCo

alternative hypotheses that satisfy this first constraint. The second constraint describes
the impact of the domain theory in PROLOG-EBL: The output hypothesis is
further constrained so that it must follow from the domain theory and the data. This
second constraint reduces the ambiguity faced by the learner when it must choose
a hypothesis. Thus, the impact of the domain theory is to reduce the effective size
of the hypothesis space and hence reduce the sample complexity of learning.
Using similar notation, we can state the type of knowledge that is required
by PROLOG-EBG for its domain theory. In particular, PROLOG-EBG assumes the
domain theory B entails the classifications of the instances in the training data:
This constraint on the domain theory B assures that an explanation can be constructed
for each positive example.
It is interesting to compare the PROLOG-EBG learning setting to the setting
for inductive logic programming (ILP) discussed in Chapter 10. In that chapter,
we discussed a generalization of the usual inductive learning task, in which background
knowledge B' is provided to the learner. We will use B' rather than B to
denote the background knowledge used by ILP, because it does not typically satisfy
the constraint given by Equation (11.3). ILP is an inductive learning system,
whereas PROLOG-EBG is deductive. ILP uses its background knowledge B' to enlarge
the set of hypotheses to be considered, whereas PROLOG-EBG uses its domain
theory B to reduce the set of acceptable hypotheses. As stated in Equation (10.2),
ILP systems output a hypothesis h that satisfies the following constraint:
Note the relationship between this expression and the constraints on h imposed
by PROLOG-EBG (given by Equations (11.1) and (11.2)). This ILP constraint on
h is a weakened form of the constraint given by Equation (11.1)-the
ILP constraint
requires only that (B' A h /\xi) k f (xi), whereas the PROLOG-EBG constraint
requires the more strict (h xi) k f (xi). Note also that ILP imposes no constraint
corresponding to the PROLOG-EBG constraint of Equation (11.2).
11.3.3 Inductive Bias in Explanation-Based Learning
Recall from Chapter 2 that the inductive bias of a learning algorithm is a set
of assertions that, together with the training examples, deductively entail subsequent
predictions made by the learner. The importance of inductive bias is
that it characterizes how the learner generalizes beyond the observed training
examples.
What is the inductive bias of PROLOG-EBG? In PROLOG-EBG the output hypothesis
h follows deductively from DAB, as described by Equation (11.2). Therefore,
the domain theory B is a set of assertions which, together with the training
examples, entail the output hypothesis. Given that predictions of the learner follow
from this hypothesis h, it appears that the inductive bias of PROLOG-EBG is simply
the domain theory B input to the learner. In fact, this is the case except for one