anytime algorithms for learning anytime classifiers saher ... - Technion

Technion - Computer Science Department - Ph.D. Thesis PHD-2008-12 - 2008
problem can be overcome by slightly modifying the learners and storing a default
class at each internal node. If classification is stopped early, the prediction is made
according to the last explored node. This modification make decision trees valid
active classifiers, i.e., classifiers that given a partially specified instance, return
either a class label or a strategy that specifies which test should be performed
next (Greiner, Grove, & Roth, 2002).
Formally, let ρc be the bound on testing costs for each case. When classifying
an example e using a tree T, we propagate e down the tree along a single path
from the root of T until a leaf is reached or classification resources are exhausted.
Let Θ(T, e) be the set of tests along this path. We denote by cost(θ) the cost
�
of administering the test θ. The testing cost of e in T is therefore tcost(T, e) =
θ∈Θ(T,e) cost(θ). Note that tests that appear several times are charged for only
once. The cost of a tree is defined as the maximal cost of classifying an example
using T. Formally,
⎧
⎨0
T is a leaf
Test-Cost(T) =
⎩cost(θroot(T)
) + max Test-Cost( ˆ T) otherwise .
ˆT ∈ Children (T)
Having decided about the model, we are interested in a learning algorithm
that can exploit additional time resources, preallocated or not, in order to improve
the induced anycost trees. In the previous Chapters we presented two
anytime algorithms for learning decision trees. The first, called LSID3 (Chapter
3), can trade computation speed for smaller and more accurate trees, and
the second, ACT (Chapter 4), can exploit extra learning time to produce more
cost-efficient trees. Neither algorithm, however, can produce trees that operate
under given classification budgets; therefore, they are inappropriate for our task.
In what follows we introduce Tree-classification AT Anycost (TATA), our proposed
anytime algorithm for learning anycost trees. TATA has three learning
components, for the three different testing-cost scenarios described in Chapter 2
: pre-contract-TATA, contract-TATA, and interruptible-TATA.
5.1 Pre-constract: Classification Budget is
Known to the Learner
The most common method for learning decision trees is TDIDT (top-down induction
of decision trees). TDIDT algorithms start with the entire set of training
examples, partition it into subsets by testing the value of an attribute, and then
recursively build subtrees. The TDIDT scheme, however, does not take into
account testing cost budgets. Therefore, we first describe TDIDT$, a modification
for adapting any TDIDT algorithm to the pre-contract scenario, and then
100