anytime algorithms for learning anytime classifiers saher ... - Technion

More documents

Recommendations

Info

Technion - Computer Science Department - Ph.D. Thesis PHD-2008-12 - 2008 Table 4.5: Average cost of classification as a percentage of the standard cost of classification for mc = 500. The first 25 rows list for each dataset the average cost over the different 4 cost-assignments, while the last 5 rows give the results for the datasets with costs from (Turney, 1995). Dataset C4.5 LSID3 IDX CSID3 EG2 DTMC ICET ACT Breast-cancer 43.3 42.0 33.4 42.7 33.9 27.8 29.1 28.0 Bupa 90.5 81.2 70.6 82.1 70.9 49.6 51.6 53.4 Car 49.3 51.9 55.2 49.5 53.7 40.1 48.5 40.1 Flare 15.0 15.0 16.8 15.4 16.8 15.0 16.5 15.0 Glass 54.1 47.2 42.3 42.3 38.9 37.8 37.5 36.6 Heart 42.3 44.8 34.5 39.2 35.0 30.2 29.9 31.0 Hepatitis 35.6 29.6 20.7 27.9 21.3 13.9 14.8 14.8 Iris 30.6 28.2 29.0 27.0 27.8 21.7 22.1 20.7 KRK 62.4 64.9 63.9 63.5 63.6 62.2 61.9 59.7 Multi-ANDOR 47.3 18.6 29.3 31.2 30.1 35.7 19.8 17.2 Monks1 26.0 20.0 25.2 25.9 26.0 53.2 23.5 22.8 Monks2 48.9 66.9 57.3 53.8 55.5 40.3 40.3 40.3 Monks3 35.0 33.5 32.7 32.6 32.6 34.6 32.2 32.5 Multiplexer 53.3 19.3 39.5 45.6 40.2 39.0 22.3 25.3 MultiXOR 60.4 26.1 43.2 51.7 44.4 42.9 43.8 32.5 Nursery 29.3 31.1 29.9 29.0 29.2 31.5 28.1 28.8 Pima 73.1 77.3 54.5 68.3 58.1 42.0 45.9 48.3 Tae 77.2 67.6 65.5 68.2 67.9 59.3 59.7 58.2 Tic-tac-toe 55.8 52.3 40.2 49.6 40.9 43.9 42.4 36.4 Titanic 76.6 59.4 66.5 69.2 66.8 51.4 56.5 51.4 Thyroid 25.5 26.5 26.5 25.3 25.2 9.7 24.5 9.7 Voting 14.4 15.6 14.2 14.4 14.2 13.9 13.9 13.6 Wine 24.8 25.7 22.4 19.7 21.0 19.8 18.4 17.6 XOR3d 86.9 54.7 61.1 71.5 61.5 64.5 58.6 60.1 XOR5 69.7 43.7 64.5 67.6 65.4 41.3 41.3 41.3 Bupa 92.4 90.2 92.2 88.2 92.2 80.3 86.0 86.6 Heart 57.0 59.6 26.7 28.8 27.3 26.3 26.3 28.9 Hepatitis 83.7 77.5 59.5 64.9 64.3 59.7 72.9 72.6 Pima 73.8 81.8 87.2 79.2 87.2 70.7 67.5 70.6 Thyroid 27.7 34.3 27.8 27.3 26.5 35.0 27.8 25.0 result in favor of ACT. Observe that DTMC, the winner when mc = 100, becomes the worst algorithm when mc = 10000. One reason for this phenomenon is that DTMC, as introduced by Ling et al. (2004), does not perform post-pruning, although doing so might improve accuracy in some domains. The above two extremes are less interesting: for the first we could use an algorithm that always outputs a tree of size 1 while for the second we could use cost-insensitive learners. The middle range, where mc ∈ {500, 1000, 5000}, 88
Technion - Computer Science Department - Ph.D. Thesis PHD-2008-12 - 2008 Table 4.6: Average cost of classification as a percentage of the standard cost of classification for mc = 1000. The first 25 rows list for each dataset the average cost over the different 4 cost-assignments, while the last 5 rows give the results for the datasets with costs from (Turney, 1995). Dataset C4.5 LSID3 IDX CSID3 EG2 DTMC ICET ACT Breast-cancer 54.3 56.7 48.0 55.4 48.6 43.6 45.2 45.5 Bupa 91.5 84.8 78.4 85.5 78.2 67.2 69.8 72.8 Car 41.4 44.3 53.8 42.8 51.4 57.2 41.2 42.1 Flare 26.0 26.0 27.1 26.3 27.1 26.0 25.9 26.0 Glass 53.9 47.2 55.0 50.9 50.2 49.2 46.1 39.7 Heart 45.0 49.5 43.0 44.2 42.8 40.3 41.2 41.4 Hepatitis 44.2 38.1 31.9 38.1 32.2 28.6 29.3 29.7 Iris 22.3 20.8 24.1 21.3 22.8 18.9 16.8 16.3 KRK 57.8 63.7 65.6 63.6 64.8 65.2 55.1 55.6 Multi-ANDOR 47.9 14.1 31.5 32.2 32.4 49.0 16.2 11.8 Monks1 16.3 12.2 15.8 16.2 16.3 52.3 15.6 12.2 Monks2 65.4 77.4 71.3 69.4 70.1 57.5 57.5 57.5 Monks3 23.8 22.9 22.5 22.3 22.3 25.5 22.1 22.0 Multiplexer 59.3 16.2 51.4 54.7 52.0 57.9 18.5 19.9 MultiXOR 62.5 18.5 51.0 56.9 52.3 56.7 49.5 21.3 Nursery 20.3 21.6 21.4 20.5 20.8 24.7 18.6 20.2 Pima 74.6 80.3 65.6 71.3 68.4 62.3 60.8 64.0 Tae 82.1 73.5 72.3 73.6 74.3 64.2 76.8 67.4 Tic-tac-toe 53.3 50.2 45.6 48.6 45.9 60.9 44.8 40.2 Titanic 72.5 62.3 65.9 67.8 66.2 64.3 60.5 61.2 Thyroid 24.3 27.3 25.4 24.1 24.1 17.7 24.1 17.7 Voting 12.7 14.1 14.8 13.0 14.3 11.8 12.0 11.8 Wine 20.3 21.9 30.0 20.7 26.8 20.6 18.8 19.1 XOR3d 89.0 49.6 75.4 77.8 75.3 84.4 70.1 66.7 XOR5 77.9 31.6 79.7 80.7 80.6 58.4 58.4 67.4 Bupa 93.0 91.2 92.8 88.5 92.8 80.8 84.3 91.2 Heart 56.0 61.1 37.4 38.9 38.3 38.6 35.4 42.3 Hepatitis 93.1 86.5 68.4 74.2 73.8 77.3 68.1 76.8 Pima 75.8 84.4 96.0 85.4 96.0 73.4 70.5 72.1 Thyroid 23.8 34.8 24.7 23.6 22.7 22.4 24.6 22.9 requires that the learner carefully balance the two types of cost. In these cases ACT has the lowest average cost and the largest number of t-test wins. Moreover, the Wilcoxon test indicates that it is superior. ICET is the second best method. As reported by Turney (1995), ICET is clearly better than the greedy methods EG2, IDX, and CSID3. Note that EG2, IDX, and CSID3, which are insensitive to misclassification cost, produced the same trees for all values of mc. These trees, however, are 89
Page 1 and 2:
Technion - Computer Science Departm
Page 3 and 4:
Page 5 and 6:
Page 7 and 8:
Page 9 and 10:
Page 11 and 12:
Page 13 and 14:
Page 15 and 16:
Page 17 and 18:
Page 19 and 20:
Page 21 and 22:
Page 23 and 24:
Page 25 and 26:
Page 27 and 28:
Page 29 and 30:
Page 31 and 32:
Page 33 and 34:
Page 35 and 36:
Page 37 and 38:
Page 39 and 40:
Page 41 and 42:
Page 43 and 44:
Page 45 and 46:
Page 47 and 48:
Page 49 and 50:
Page 51 and 52:
Page 53 and 54: Technion - Computer Science Departm
Page 103: Technion - Computer Science Departm
Page 155 and 156:
Page 157 and 158:
Page 159 and 160:
Page 161 and 162:
Page 163 and 164:
Page 165 and 166:
Page 167 and 168:
Page 169 and 170:
Page 171 and 172:
Page 173 and 174:
Page 175 and 176:
Page 177 and 178:
Page 179 and 180:
Page 181 and 182:
Page 183 and 184:
Page 185 and 186:
Page 187 and 188:
Page 189 and 190:
Page 191 and 192:
show all

anytime algorithms for learning anytime classifiers saher ... - Technion

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?