Machine Learning - DISCo

This Foil-Gain function has a straightforward interpretation in terms of
information theory. According to information theory, - log2 -/& is the minimum
number of bits needed to encode the classification of an arbitrary positive binding
among the bindings covered by rule R. Similarly, -log2 A is the number
of bits required if the binding is one of those covered by rule R'. Since t is
just the number of positive bindings covered by R that remain covered by R',
Foil-Gain(L, R) can be seen as the reduction due to L in the total number of
bits needed to encode the classification of all positive bindings of R.
10.5.3 Learning Recursive Rule Sets
In the above discussion, we ignored the possibility that new literals added to the
rule body could refer to the target predicate itself (i.e., the predicate occurring
in the rule head). However, if we include the target predicate in the input list of
Predicates, then FOIL will consider it as well when generating candidate literals.
This will allow it to form recursive rules-rules that use the same predicate in
the body and the head of the rule. For instance, recall the following rule set that
provides a recursive definition of the Ancestor relation.
IF Parent (x, y) THEN Ancestor(x, y)
IF Parent (x, z) A Ancestor(z, y) THEN Ancestor@, y)
Given an appropriate set of training examples, these two rules can be learned
following a trace similar to the one above for GrandDaughter. Note the second
rule is among the rules that are potentially within reach of FOIL'S search, provided
Ancestor is included in the list Predicates that determines which predicates may
be considered when generating new literals. Of course whether this particular
rule would be learned or not depends on whether these particular literals outscore
competing candidates during FOIL'S greedy search for increasingly specific rules.
Cameron-Jones and Quinlan (1993) discuss several examples in which FOIL has
successfully discovered recursive rule sets. They also discuss important subtleties
that arise, such as how to avoid learning rule sets that produce infinite recursion.
10.5.4 Summary of FOIL
To summarize, FOIL extends the sequential covering algorithm of CN2 to handle
the case of learning first-order rules similar to Horn clauses. To learn each rule
FOIL performs a general-to-specific search, at each step adding a single new literal
to the rule preconditions. The new literal may refer to variables already mentioned
in the rule preconditions or postconditions, and may introduce new variables as
well. At each step, it uses the Foil-Gain function of Equation (10.1) to select
among the candidate new literals. If new literals are allowed to refer to the target
predicate, then FOIL can, in principle, learn sets of recursive rules. While this introduces
the complexity of avoiding rule sets that result in infinite recursion, FOIL
has been demonstrated to successfully learn recursive rule sets in several cases.