anytime algorithms for learning anytime classifiers saher ... - Technion

More documents

Recommendations

Info

Technion - Computer Science Department - Ph.D. Thesis PHD-2008-12 - 2008 problem can be overcome by slightly modifying the learners and storing a default class at each internal node. If classification is stopped early, the prediction is made according to the last explored node. This modification make decision trees valid active classifiers, i.e., classifiers that given a partially specified instance, return either a class label or a strategy that specifies which test should be performed next (Greiner, Grove, & Roth, 2002). Formally, let ρc be the bound on testing costs for each case. When classifying an example e using a tree T, we propagate e down the tree along a single path from the root of T until a leaf is reached or classification resources are exhausted. Let Θ(T, e) be the set of tests along this path. We denote by cost(θ) the cost � of administering the test θ. The testing cost of e in T is therefore tcost(T, e) = θ∈Θ(T,e) cost(θ). Note that tests that appear several times are charged for only once. The cost of a tree is defined as the maximal cost of classifying an example using T. Formally, ⎧ ⎨0 T is a leaf Test-Cost(T) = ⎩cost(θroot(T) ) + max Test-Cost( ˆ T) otherwise . ˆT ∈ Children (T) Having decided about the model, we are interested in a learning algorithm that can exploit additional time resources, preallocated or not, in order to improve the induced anycost trees. In the previous Chapters we presented two anytime algorithms for learning decision trees. The first, called LSID3 (Chapter 3), can trade computation speed for smaller and more accurate trees, and the second, ACT (Chapter 4), can exploit extra learning time to produce more cost-efficient trees. Neither algorithm, however, can produce trees that operate under given classification budgets; therefore, they are inappropriate for our task. In what follows we introduce Tree-classification AT Anycost (TATA), our proposed anytime algorithm for learning anycost trees. TATA has three learning components, for the three different testing-cost scenarios described in Chapter 2 : pre-contract-TATA, contract-TATA, and interruptible-TATA. 5.1 Pre-constract: Classification Budget is Known to the Learner The most common method for learning decision trees is TDIDT (top-down induction of decision trees). TDIDT algorithms start with the entire set of training examples, partition it into subsets by testing the value of an attribute, and then recursively build subtrees. The TDIDT scheme, however, does not take into account testing cost budgets. Therefore, we first describe TDIDT$, a modification for adapting any TDIDT algorithm to the pre-contract scenario, and then 100
Technion - Computer Science Department - Ph.D. Thesis PHD-2008-12 - 2008 ρ c = $30 ? $30 a5 a2 ρ c = $60 candidate splits a1 a2 a a4 a5 a6 40 0 20 25 0 10 ρ c = $20 a6 $40 a5 a2 ρ c = $20 ρ c = $60 Figure 5.1: Exemplifying the recursive invocation of TDIDT$. The initial ρ c was $60 (left-hand figure). Since two tests (a2 and a5) with a total cost of $30 have been used, their current cost is zero and the remaining budget for the TDIDT$ process is $30. a1, whose cost is $40, is filtered out. Assume that, among the remaining candidates, a6 is chosen. The subtrees of the new split (right-hand figure) are built recursively with a budget of $20. introduce the pre-contract component in the TATA framework, which adopts TDIDT$. 5.1.1 Top-down Induction of Anycost Trees Any decision tree can serve as an anycost classifier by simply storing a default classification at each node and predicting according to the last explored node. Expensive tests, in this case, may block their subtrees and make them useless. In the pre-contract setup, ρ c , the maximal testing cost, is known in advance and thus the tree learner can avoid exceeding ρ c by considering only tests whose costs are lower. When a choice is made to split on a test θ, the cost bound for building the subtree under it is decreased by cost(θ). Figure 5.1 exemplifies the recursive invocation of TDIDT$. The initial ρ c was $60 (left-hand figure). Since two tests (a2 and a5) with a total cost of $30 have been used, their current cost is zero and the remaining budget for the TDIDT$ process is $30. a1, whose cost is $40, is filtered out. Assume that, among the remaining candidates, a6 is chosen. The subtrees of the new split (right-hand figure) are built recursively with a budget of $20. Figure 5.2 formalizes the modified TDIDT procedure, denoted TDIDT$. TDIDT$ is designed to search in the space of trees whose test-cost is at most 101
Page 1 and 2:
Technion - Computer Science Departm
Page 3 and 4:
Page 5 and 6:
Page 7 and 8:
Page 9 and 10:
Page 11 and 12:
Page 13 and 14:
Page 15 and 16:
Page 17 and 18:
Page 19 and 20:
Page 21 and 22:
Page 23 and 24:
Page 25 and 26:
Page 27 and 28:
Page 29 and 30:
Page 31 and 32:
Page 33 and 34:
Page 35 and 36:
Page 37 and 38:
Page 39 and 40:
Page 41 and 42:
Page 43 and 44:
Page 45 and 46:
Page 47 and 48:
Page 49 and 50:
Page 51 and 52:
Page 53 and 54:
Page 55 and 56:
Page 57 and 58:
Page 59 and 60:
Page 61 and 62:
Page 63 and 64:
Page 65 and 66: Technion - Computer Science Departm
Page 115: Technion - Computer Science Departm
Page 167 and 168:
Page 169 and 170:
Page 171 and 172:
Page 173 and 174:
Page 175 and 176:
Page 177 and 178:
Page 179 and 180:
Page 181 and 182:
Page 183 and 184:
Page 185 and 186:
Page 187 and 188:
Page 189 and 190:
Page 191 and 192:
show all

anytime algorithms for learning anytime classifiers saher ... - Technion

Create successful ePaper yourself

Delete template?

Save as template?