anytime algorithms for learning anytime classifiers saher ... - Technion

Technion - Computer Science Department - Ph.D. Thesis PHD-2008-12 - 2008
Chapter 3
Contract Anytime Learning of
Accurate Trees
Assume that a medical center has decided to use medical records of previous
patients in order to build an automatic diagnostic system for a particular disease.
The center applies the C4.5 algorithm on thousands of records, and after few
seconds receives a decision tree. During the coming months, or even years, the
same induced decision tree will be used to predict whether patients have or do
not have the disease. Obviously, the medical center is willing to wait much longer
to obtain a better tree—either more accurate or more comprehensible.
Consider also a planning agent that has to learn a decision tree from a given
set of examples, while the time at which the model will be needed by the agent is
not known in advance. In this case, the agent would like the learning procedure
to learn the best tree it can until it is interrupted and queried for a solution.
In both of the above scenarios, the learning algorithm is expected to exploit
additional time allocation to produce a better tree. In the first case, the additional
time is allocated in advance. In the second, it is not. Similar resource-bounded
reasoning situations may occur in many real-life applications such as game playing,
planning, stock trading and e-mail filtering. In this Chapter, we introduce a
framework for exploiting extra time, preallocated or not, in order to learn smaller
and more accurate trees (Esmeir & Markovitch, 2007b).
3.1 Top-down Induction of Decision Trees
TDIDT (top-down induction of decision trees) methods start from the entire set
of training examples, partition it into subsets by testing the value of an attribute,
and then recursively call the induction algorithm for each subset. Figure 3.1
formalizes the basic algorithm for TDIDT. We focus first on consistent trees for
19