Machine Learning - DISCo

(e.g., neural network) fits the data based solely on the description of the
hypothesis and data, independent of the task domain under study. In contrast,
this formulation allows the domain-specific background information B to
become part of the definition of "fit." In particular, h fits the training example
(xi, f (xi)) as long as f (xi) follows deductively from B A h A xi.
0 By incorporating background information B, this formulation invites learning
methods that use this background information to guide the search for h,
rather than merely searching the space of syntactically legal hypotheses.
The inverse resolution procedure described in the following section uses
background knowledge in this fashion.
At the same time, research on inductive logic programing following this
formulation has encountered several practical difficulties.
a The requirement @'(xi, f (xi)) E D) (B A h A xi) t f (xi) does not naturally
accommodate noisy training data. The problem is that this expression does
not allow for the possibility that there may be errors in the observed description
of the instance xi or its target value f (xi). Such errors can produce
an inconsistent set of constraints on h. Unfortunately, most formal logic
frameworks completely lose their ability to distinguish between truth and
falsehood once they are given inconsistent sets of assertions.
0 The language of first-order logic is so expressive, and the number of hypotheses
that satisfy (V(xi, f (xi)) E D) (B A h A xi) t f (xi) is SO large,
that the search through the space of hypotheses is intractable in the general
case. Much recent work has sought restricted forms of first-order expressions,
or additional second-order knowledge, to improve the tractability of
the hypothesis space search.
0 Despite our intuition that background knowledge B should help constrain
the search for a hypothesis, in most ILP systems (including all discussed
in this chapter) the complexity of the hypothesis space search increases as
background knowledge B is increased. (However, see Chapters 11 and 12 for
algorithms that use background knowledge to decrease rather than increase
sample complexity).
In the following section, we examine one quite general inverse entailment
operator that constructs hypotheses by inverting a deductive inference rule.
10.7 INVERTING RESOLUTION
A general method for automated deduction is the resolution rule introduced by
Robinson (1965). The resolution rule is a sound and complete rule for deductive
inference in first-order logic. Therefore, it is sensible to ask whether we can invert
the resolution rule to form an inverse entailment operator. The answer is yes, and
it is just this operator that forms the basis of the CIGOL program introduced by
Muggleton and Buntine (1988).