anytime algorithms for learning anytime classifiers saher ... - Technion

Technion - Computer Science Department - Ph.D. Thesis PHD-2008-12 - 2008
Table 3.2: Characteristics of the datasets used
Attributes Max
Dataset Instances Nom. (bin.) Num. domain Classes
Automobile - Make 160 10 (4) 15 8 22
Automobile - Sym. 160 10 (4) 15 22 7
Balance Scale 625 4 (0) 0 5 3
Breast Cancer 277 9 (3) 0 13 2
Connect-4 68557 42 (0) 0 3 3
Corral 32 6 (6) 0 2 2
Glass 214 0 (0) 9 - 7
Iris 150 0 (0) 4 - 3
Monks-1 124+432 6 (2) 0 4 2
Monks-2 169+432 6 (2) 0 4 2
Monks-3 122+432 6 (2) 0 4 2
Mushroom 8124 22 (4) 0 12 2
Solar Flare 323 10 (5) 0 7 4
Tic-Tac-Toe 958 9 (0) 0 3 2
Voting 232 16 (16) 0 2 2
Wine 178 0 (0) 13 - 3
Zoo 101 16 (15) 0 6 7
Numeric XOR 3D 200 0 (0) 6 - 2
Numeric XOR 4D 500 0 (0) 8 - 2
Multiplexer-20 615 20 (20) 0 2 2
Multiplex-XOR 200 11 (11) 0 2 2
XOR-5 200 10 (10) 0 2 2
XOR-5 10% noise 200 10 (10) 0 2 2
XOR-10 10000 20 (20) 0 2 2
ones. 10 Several commonly used UCI datasets were included (among the 17) to
allow comparison with results reported in literature. Table 3.2 summarizes the
characteristics of these datasets while Appendix A gives more detailed descriptions.
To compare the performance of the different algorithms, we will consider
two evaluation criteria over decision trees: (1) generalization accuracy, measured
by the ratio of the correctly classified examples in the testing set, and (2) tree
size, measured by the number of non-empty leaves in the tree.
3.7.2 Fixed Time Comparison
Our first set of experiments compares ID3, C4.5 with its default parameters, ID3k(k
= 2), LSID3(r = 5) and LSID3-p(r = 5). We used our own implementation
for all algorithms, where the results of C4.5 were validated with WEKA’s implementation
(Witten & Frank, 2005). We first discuss the results for the consistent
trees and continue by analyzing the findings when pruning is applied.
10 The artificial datasets are available at http://www.cs.technion.ac.il/∼esaher/publications/datasets.
46