anytime algorithms for learning anytime classifiers saher ... - Technion

Technion - Computer Science Department - Ph.D. Thesis PHD-2008-12 - 2008
Average Size
Average Accuracy
180
160
140
120
100
80
60
40
20
IIDT(0.1)
Skewing
Sequential Skewing
0
0 1 2 3 4 5 6 7
100
90
80
70
60
50
Time [sec]
IIDT(0.1)
Skewing
Sequential Skewing
Bagging-ID3
Bagging-LSID3
0 1 2 3 4 5 6 7
Time [sec]
Figure 3.34: Anytime behavior of modern learners on the Multiplexer-20 dataset
its results separately later in this section.
The graphs for the first 2 problems, which are known to be hard, show that
IIDT clearly outperforms the other methods both in terms of tree size and accuracy.
In both cases IIDT reaches almost perfect accuracy (99%), while bagging-
ID3 and skewing topped at 55% for the first problem and 75% for the second.
The inferior performance of bagging-ID3 on the XOR-5 and Multiplexer-20
tasks is not surprising. The trees that form the committee were induced greedily
and hence could not discover these difficult concepts, even when they were combined.
Similar results were obtained when running bagging over C4.5 and RTG.
However, when our LSID3(r = 1) was used as a base learner, performance was
significantly better than that of the greedy committees. Still, IIDT performed
significantly better than bagging-LSID3, indicating that for difficult concepts, it
is better to invest more resources for improving a single tree than for adding more
trees of lower quality to the committee.
67