Machine Learning - DISCo

Hypothesis Space
1
Hypotheses thatfit
training data
equally well
FIGURE 12.9
Hypothesis space search in FOCL. FOCL augments the set of search operators used by FOIL. Whereas
FOIL considers adding a single new literal at each step, FOIL also considers adding multiple literals
derived from the domain theory.
the bias of the purely inductive FOIL program, which is a preference for shorter
hypotheses.
FOCL has been shown to generalize more accurately than the purely inductive
FOIL algorithm in a number of application domains in which an imperfect domain
theory is available. For example, Pazzani and Kibler (1992) explore learning
the concept "legal chessboard positions." Given 60 training examples describing
30 legal and 30 illegal endgame board positions, FOIL achieved an accuracy of
86% over an independent set of test examples. FOCL was given the same 60 training
examples, along with an approximate domain theory with an accuracy of 76%.
FOCL produced a hypothesis with generalization accuracy of 94%-less than half
the error rate of FOIL. Similar results have been obtained in other domains. For
example, given 500 training examples of telephone network problems and their
diagnoses from the telephone company NYNEX, FOIL achieved an accuracy of
90%, whereas FOCL reached an accuracy of 98% when given the same training
data along with a 95% accurate domain theory.
12.6 STATE OF THE ART
The methods presented in this chapter are only a sample of the possible approaches
to combining analytical and inductive learning. While each of these methods has
been demonstrated to outperform purely inductive learning methods in selected
domains, none of these has been thoroughly tested or proven across a large variety
of problem domains. The topic of combining inductive and analytical learning
remains a very active research area.