Machine Learning - DISCo

In the case of noise-free training data, FOIL may continue adding new literals
to the rule until it covers no negative examples. To handle noisy data, the search
is continued until some tradeoff occurs between rule accuracy, coverage, and
complexity. FOIL uses a minimum description length approach to halt the growth
of rules, in which new literals are added only when their description length is
shorter than the description length of the training data they explain. The details
of this strategy are given in Quinlan (1990). In addition, FOIL post-prunes each
rule it learns, using the same rule post-pruning strategy used for decision trees
(Chapter 3).
10.6 INDUCTION AS INVERTED DEDUCTION
A second, quite different approach to inductive logic programming is based on
the simple observation that induction is just the inverse of deduction! In general,
machine learning involves building theories that explain the observed data. Given
some data D and some partial background knowledge B, learning can be described
as generating a hypothesis h that, together with B, explains D. Put more precisely,
assume as usual that the training data D is a set of training examples, each of
the form (xi, f (xi)). Here xi denotes the ith training instance and f (xi) denotes
its target value. Then learning is the problem of discovering a hypothesis h, such
that the classification f (xi) of each training instance xi follows deductively from
the hypothesis h, the description of xi, and any other background knowledge B
known to the system.
(V(xi, f (xi)) E D) (B Ah A xi) f (xi) (10.2)
The expression X F Y is read "Y follows deductively from X," or alternatively
"X entails Y." Expression (10.2) describes the constraint that must be satisfied
by the learned hypothesis h; namely, for every training instance xi, the target
classification f (xi) must follow deductively from B, h, and xi.
As an example, consider the case where the target concept to be learned is
"pairs of people (u, v) such that the child of u is v," represented by the predicate
Child(u, v). Assume we are given a single positive example Child(Bob, Sharon),
where the instance is described by the literals Male(Bob), Female(Sharon), and
Father(Sharon, Bob). Furthermore, suppose we have the general background
knowledge Parent (u, v) t Father (u, v). We can describe this situation in the
terms of Equation (10.2) as follows:
xi : Male(Bob), Female(Sharon), Father(Sharon, Bob)
f (xi) :
Child(Bob, Sharon)
In this case, two of the many hypotheses that satisfy the constraint (B Ah A xi) t-
f (xi) are
hl : Child(u, v) t Father(v, u)
h2 : Child(u, v) t Parent (v, u)