anytime algorithms for learning anytime classifiers saher ... - Technion

Technion - Computer Science Department - Ph.D. Thesis PHD-2008-12 - 2008
Chapter 4
Anytime Learning of
Cost-sensitive Trees
Assume, for example, that a medical center has decided to use machine learning
techniques to induce a diagnostic tool from records of previous patients. The
center aims to obtain a comprehensible model, with low expected test costs and
high expected accuracy. Moreover, in many cases there are costs associated with
the predictive errors. In such a scenario, the task of the inducer is to produce a
model with low expected test costs and low expected misclassification costs.
In this chapter we build on the LSID3 algorithm, presented in Chapter 3, and
introduce ACT (Anytime learning of Cost-sensitive Trees), our proposed anytime
framework for induction of cost-sensitive decision trees (Esmeir & Markovitch,
2007a). ACT attempts to minimize the total cost of classification, that is the
sum of testing costs and misclassification costs. ACT takes the same sampling
approach as LSID3. The three major components of LSID3 that need to be
replaced in order to adapt it for cost-sensitive problems are: (1) sampling the
space of trees, (2) evaluating a tree, and (3) pruning a tree.
4.1 Biasing the Sample Towards Low-cost Trees
LISD3 uses SID3 to bias the samples towards small trees. In ACT, however,
we would like to bias our sample towards low-cost trees. For this purpose, we
designed a stochastic version of the EG2 algorithm, which attempts to build low
cost trees greedily (Nunez, 1991). In EG2, a tree is built top-down, and the test
that maximizes ICF is chosen for splitting a node, where,
ICF (θ) = 2∆I(θ) − 1
(cost (θ) + 1) w.
71