anytime algorithms for learning anytime classifiers saher ... - Technion

Technion - Computer Science Department - Ph.D. Thesis PHD-2008-12 - 2008
Table 5.1: Characteristics of the datasets used to evaluate TATA
Attributes Max
Dataset Instances Nom. (bin.) Num. domain Classes
Breast Cancer 277 9 (3) 0 13 2
Bupa 345 0 (0) 5 - 2
Car 1728 6 (0) 0 4 4
Flare 323 10 (5) 0 7 4
Glass 214 0 (0) 9 - 7
Heart 296 8(4) 5 4 2
Hepatitis 154 13(13) 6 2 2
Iris 150 0 (0) 4 - 3
KRK 28056 6(0) 0 8 17
Monks-1 124+432 6 (2) 0 4 2
Monks-2 169+432 6 (2) 0 4 2
Monks-3 122+432 6 (2) 0 4 2
Multiplexer-20 615 20 (20) 0 2 2
Multi-XOR 200 11 (11) 0 2 2
Multi-AND-OR 200 11 (11) 0 2 2
Nursery 8703 8(8) 0 5 5
Pima 768 0(0) 8 - 2
TAE 151 4(1) 1 26 3
Tic-Tac-Toe 958 9 (0) 0 3 2
Titanic 2201 3(2) 0 4 2
Thyroid 3772 15(15) 5 2 3
Voting 232 16 (16) 0 2 2
Wine 178 0 (0) 13 - 3
XOR 3D 200 0 (0) 6 - 2
XOR-5 200 10 (10) 0 2 2
administer any test and thus their performance is identical. At the other end,
when ρ ≥ ρc max , the attribute costs are actually not a constraint. In this case
TATA(r = 5) performed best, confirming the results reported in Chapter 4 when
misclassification costs were dominant. The more interesting ρc values are those
in between. Table 5.2 lists the normalized area under the misclassification cost
curve over the range [33%−99%]ρc max. Confirming the curves, the results indicate
that TATA(r = 5) has the best overall performance. The Wilcoxon test (Demsar,
2006), which compares classifiers over multiple datasets, finds TATA(r = 5) to
be significantly better than all the other algorithms.
As expected, all five algorithms improve with the increase in ρc because they
can use more features. For ρc values slightly larger than ρc min we can see that EG2,
which is cost-sensitive, performs better than C4.5. The reason is that EG2 takes
into account attribute costs and hence will prefer lower cost attributes. With the
increase in ρc and the relaxation of cost constraints, C4.5 becomes better than
EG2.
112