Machine Learning - DISCo

more restricted than general Horn clauses, because the literals are not pennitted
to contain function symbols (this reduces the complexity of the hypothesis space
search). Second, FOIL rules are more expressive than Horn clauses, because the
literals appearing in the body of the rule may be negated. FOIL has been applied
to a variety of problem domains. For example, it has been demonstrated to learn a
recursive definition of the QUICKSORT algorithm and to learn to discriminate legal
from illegal chess positions.
The FOIL algorithm is summarized in Table 10.4. Notice the outer loop
corresponds to a variant of the SEQUENTIAL-COVERING algorithm discussed earlier;
that is, it learns new rules one at a time, removing the positive examples covered by
the latest rule before attempting to learn the next rule. The inner loop corresponds
to a variant of our earlier LEARN-ONE-RULE algorithm, extended to accommodate
first-order rules. Note also there are a few minor differences between FOIL and
these earlier algorithms. In particular, FOIL seeks only rules that predict when
the target literal is True, whereas our earlier algorithm would seek both rules
that predict when it is True and rules that predict when it is False. Also, FOIL
performs a simple hillclimbing search rather than a beam search (equivalently, it
uses a beam of width one).
The hypothesis space search performed by FOIL is best understood by view-
ing it hierarchically. Each iteration through FOIL'S outer loop adds a new rule to
its disjunctive hypothesis, Learned~ules. The effect of each new rule is to gen-
--
FOIL(Target-predicate, Predicates, Examples)
Pos c those Examples for which the Target-predicate is True
Neg c those Examples for which the Target-predicate is False
while Pos, do
Learn a NewRule
New Rule t the rule that predicts Target-predicate with no preconditions
NewRuleNeg t Neg
while NewRuleNeg, do
Add a new literal to specialize New Rule
Candidateliterals t generate candidate new literals for NewRule, based on
Predicates
Bestliteralt argmax Foil-Gain(L,NewRule)
LECandidateliterals
add Bestliteral to preconditions of NewRule
NewRuleNeg c subset of NewRuleNeg that satisfies NewRule preconditions
Learned~ul es c Learned-rules + NewRule
Pos t Pos - {members of Pos covered by NewRule)
Return Learned-rules
TABLE 10.4
The basic FOIL algorithm. The specific method for generating Candidateliterals and the defini- ~
tion of Foil-Gain are given in the text. This basic algorithm can be modified slightly to better
accommodate noisy data, as described in the text.