Machine Learning - DISCo

call LEMMA-ENUMERATOR. The LEMMA-ENUMERATOR algorithm simply enumerates
all proof trees that conclude the target concept based on assertions in the domain
theory B. For each such proof tree, LEMMA-ENUMERATOR calculates the weakest
preimage and constructs a Horn clause, in the same fashion as PROLOG-EBG. The
only difference between LEMMA-ENUMERATOR and PROLOG-EBG is that LEMMA-
ENUMERATOR ignores the training data and enumerates all proof trees.
Notice LEMMA-ENUMERATOR will output a superset of the Horn clauses output
by PROLOG-EBG. Given this fact, several questions arise. First, if its hypotheses
follow from the domain theory alone, then what is the role of training data in
PROLOG-EBG? The answer is that training examples focus the PROLOG-EBG algorithm
on generating rules that cover the distribution of instances that occur in
practice. In our original chess example, for instance, the set of all possible lemmas
is huge, whereas the set of chess positions that occur in normal play is only a
small fraction of those that are syntactically possible. Therefore, by focusing only
on training examples encountered in practice, the program is likely to develop a
smaller, more relevant set of rules than if it attempted to enumerate all possible
lemmas about chess.
The second question that arises is whether PROLOG-EBG can ever learn a
hypothesis that goes beyond the knowledge that is already implicit in the domain
theory. Put another way, will it ever learn to classify an instance that could not
be classified by the original domain theory (assuming a theorem prover with
unbounded computational resources)? Unfortunately, it will not. If B F h, then
any classification entailed by h will also be entailed by B. Is this an inherent
limitation of analytical or deductive learning methods? No, it is not, as illustrated
by the following example.
To produce an instance of deductive learning in which the learned hypothesis
h entails conclusions that are not entailed by B, we must create an example where
B y h but where D A B F h (recall the constraint given by Equation (11.2)).
One interesting case is when B contains assertions such as "If x satisfies the
target concept, then so will g(x)." Taken alone, this assertion does not entail the
classification of any instances. However, once we observe a positive example, it
allows generalizing deductively to other unseen instances. For example, consider
learning the PlayTennis target concept, describing the days on which our friend
Ross would like to play tennis. Imagine that each day is described only by the
single attribute Humidity, and the domain theory B includes the single assertion
"If Ross likes to play tennis when the humidity is x, then he will also like to play
tennis when the humidity is lower than x," which can be stated more formally as
(Vx)
IF ((PlayTennis = Yes) t (Humidity = x))
THEN ((PlayTennis = Yes) t (Humidity 5 x))
Note that this domain theory does not entail any conclusions regarding which
instances are positive or negative instances of PlayTennis. However, once the
learner observes a positive example day for which Humidity = .30, the domain
theory together with this positive example entails the following general hypothe-