anytime algorithms for learning anytime classifiers saher ... - Technion

More documents

Recommendations

Info

Technion - Computer Science Department - Ph.D. Thesis PHD-2008-12 - 2008 · · · Enew ← Get-New-Examples() Foreach e ∈ Enew Add-Example(T, E, A, e) E ← E ∪ {e} Return T Figure 3.17: I-IIDT (only the lines added to IIDT are listed) Procedure TS-Add-Example(T, E, A, e) l ← Leaf-Of(e) Tl ← ID3(T,Examples-At(l, T, E),Att(l)) replace l with Tl ∆size ← Tree-Size(Tl) − 1 Foreach p ancestor of Tl Tree-Size(p) ← Tree-Size(p) + ∆size Figure 3.18: Incorporating a new example by I-IIDT-TS Our first method for tree update is intended for IIDT-TS. Recall that IIDT retains for each node y the set of examples belongs to it, Ey. Given a new example e, and the current tree T, we search for the leaf l that e belongs to. If the label of l differs from that of e, we replace l by a consistent tree, built from El ∪ {e}. This tree can be quickly obtained by invoking ID3. Such an update, that replaces a leaf l with a subtree, increases the size of the subtrees under all the nodes along the path from the root to l. Consequently, the expected benefit from rebuilding these subtrees increases. On the other hand, the expected cost, which depends on the number of examples, will increase as well. The selection mechanism of I-IIDT which prefers subtrees with high expected utility will take both changes into consideration. We refer to this variant as I-IIDT-TS. Figure 3.18 lists I-IIDT-TS. The second method is designed for IIDT-EE. With the possibility to maintain inconsistent trees, adding a new example to the existing tree becomes as simple as finding the relevant leaf and updating the expected error of this leaf and of its ancestors. Clearly, a new example that is consistent with the current tree reduces the expected error, while an inconsistent example increases it. This effect is the strongest in the leaf and diminishes for shallower subtrees. We refer to this variant as I-IIDT-EE. Figure 3.19 formalizes I-IIDT-EE. 44
Technion - Computer Science Department - Ph.D. Thesis PHD-2008-12 - 2008 Procedure EE-Add-Example(T, E, A, e) l ← Leaf-Of(e) error ←Expected-Error(Ex(l)) El ← Examples-At(l, T, E) ∪ {e} error ′ ←Expected-Error(El) ∆error ← error − error ′ Foreach p ancestor of l Expected-Error(Examples-At(p, T, E)) ← Expected-Error(Examples-At(p, T, E)) + ∆error Figure 3.19: Incorporating a new example by I-IIDT-EE 3.7 Empirical Evaluation A variety of experiments were conducted to test the performance and behavior of the proposed anytime algorithms. First we describe our experimental methodology and explain its motivation. We then present and discuss our results. 3.7.1 Experimental Methodology We start our experimental evaluation by comparing our contract algorithm, given a fixed resource allocation, with the basic decision tree induction algorithms. We then compare the anytime behavior of our contract algorithm to that of fixed lookahead. Next we examine the anytime behavior of our interruptible algorithm. Finally, we compare its performance to several modern decision tree induction methods. Following the recommendations of Bouckaert (2003), 10 runs of a 10-fold crossvalidation experiment were conducted for each dataset and the reported results averaged over the 100 individual runs. 8 For the Monks datasets, which were originally partitioned into a training set and a testing set, we report the results on the original partitions. Due to the stochastic nature of LSID3, the reported results in these cases are averaged over 10 different runs. In order to evaluate the studied algorithms, we used 17 datasets taken from the UCI repository (Asuncion & Newman, 2007). 9 Because greedy learners perform quite well on easy tasks, we looked for problems that hide hard concepts so the advantage of our proposed methods will be emphasized. The UCI repository, nevertheless, contains only few such tasks. Therefore, we added 7 artificial 8An exception was the Connect-4 dataset, for which only one run of 10-fold CV was conducted because of its enormous size. 9Due to the time-intensiveness of the experiments, we limited ourselves to 17 UCI problems. 45
Page 1 and 2:
Technion - Computer Science Departm
Page 3 and 4:
Page 5 and 6:
Page 7 and 8:
Page 9 and 10: Technion - Computer Science Departm
Page 59: Technion - Computer Science Departm
Page 111 and 112:
Page 113 and 114:
Page 115 and 116:
Page 117 and 118:
Page 119 and 120:
Page 121 and 122:
Page 123 and 124:
Page 125 and 126:
Page 127 and 128:
Page 129 and 130:
Page 131 and 132:
Page 133 and 134:
Page 135 and 136:
Page 137 and 138:
Page 139 and 140:
Page 141 and 142:
Page 143 and 144:
Page 145 and 146:
Page 147 and 148:
Page 149 and 150:
Page 151 and 152:
Page 153 and 154:
Page 155 and 156:
Page 157 and 158:
Page 159 and 160:
Page 161 and 162:
Page 163 and 164:
Page 165 and 166:
Page 167 and 168:
Page 169 and 170:
Page 171 and 172:
Page 173 and 174:
Page 175 and 176:
Page 177 and 178:
Page 179 and 180:
Page 181 and 182:
Page 183 and 184:
Page 185 and 186:
Page 187 and 188:
Page 189 and 190:
Page 191 and 192:
show all

anytime algorithms for learning anytime classifiers saher ... - Technion

Create successful ePaper yourself

Delete template?

Save as template?