Machine Learning - DISCo

50 MACHINE LEARNING
2.8. In this chapter, we commented that given an unbiased hypothesis space (the power
set of the instances), the learner would find that each unobserved instance would
match exactly half the current members of the version space, regardless of which
training examples had been observed. Prove this. In particular, prove that for any
instance space X, any set of training examples D, and any instance x E X not present
in D, that if H is the power set of X, then exactly half the hypotheses in VSH,D will
classify x as positive and half will classify it as negative.
2.9. Consider a learning problem where each instance is described by a conjunction of
n boolean attributes a1 . . .a,. Thus, a typical instance would be
(al = T) A (az = F) A . . . A (a, = T)
Now consider a hypothesis space H in which each hypothesis is a disjunction of
constraints over these attributes. For example, a typical hypothesis would be
Propose an algorithm that accepts a sequence of training examples and outputs
a consistent hypothesis if one exists. Your algorithm should run in time that is
polynomial in n and in the number of training examples.
2.10. Implement the FIND-S algorithm. First verify that it successfully produces the trace in
Section 2.4 for the Enjoysport example. Now use this program to study the number
of random training examples required to exactly learn the target concept. Implement
a training example generator that generates random instances, then classifies them
according to the target concept:
(Sunny, Warm, ?, ?, ?, ?)
Consider training your FIND-S program on randomly generated examples and measuring
the number of examples required before the program's hypothesis is identical
to the target concept. Can you predict the average number of examples required?
Run the experiment at least 20 times and report the mean number of examples required.
How do you expect this number to vary with the number of "?" in the
target concept? How would it vary with the number of attributes used to describe
instances and hypotheses?
REFERENCES
Bruner, J. S., Goodnow, J. J., & Austin, G. A. (1957). A study of thinking. New York: John Wiey
& Sons.
Buchanan, B. G. (1974). Scientific theory formation by computer. In J. C. Simon (Ed.), Computer
Oriented Learning Processes. Leyden: Noordhoff.
Gunter, C. A., Ngair, T., Panangaden, P., & Subramanian, D. (1991). The common order-theoretic
structure of version spaces and ATMS's. Proceedings of the National Conference on Artijicial
Intelligence (pp. 500-505). Anaheim.
Haussler, D. (1988). Quantifying inductive bias: A1 learning algorithms and Valiant's learning framework.
Artijicial Intelligence, 36, 177-221.
Hayes-Roth, F. (1974). Schematic classification problems and their solution. Pattern Recognition, 6,
105-113.
Hirsh, H. (1990). Incremental version space merging: A general framework for concept learning.
Boston: Kluwer.