anytime algorithms for learning anytime classifiers saher ... - Technion

Technion - Computer Science Department - Ph.D. Thesis PHD-2008-12 - 2008
Average Size
Average Accuracy
100
90
80
70
60
50
40
30
20
10
IIDT(0.1)
Skewing
Sequential Skewing
0
0 0.05 0.1 0.15 0.2 0.25
100
90
80
70
60
50
Time [sec]
IIDT(0.1)
Skewing
Sequential Skewing
Bagging-ID3
Bagging-LSID3
0 0.05 0.1 0.15 0.2 0.25
Time [sec]
Figure 3.33: Anytime behavior of modern learners on the XOR-5 dataset
Empirical Comparison
We used our own implementation for IIDT, skewing, and bagging, and the commercial
version for GATree. 15 The skewing and sequential skewing versions were
run with linearly increasing parameters. The generalized skewing algorithm was
run with exponentially increasing parameters. The performance of the ensemble
method was tested for exponentially increasing committee sizes (1, 2, 4, 8, . . .).
Figures 3.33, 3.34, and 3.35 compare IIDT to bagging with ID3 as a base
learner, bagging with LSID3(r = 1), and skewing on the XOR-5, Multiplexer-20,
and Tic-tac-toe tasks respectively. Note that the results for ID3 are identical
to those of bagging-ID3 with a single tree in the committee and hence are not
plotted independently. Since GATree was run on a different machine, we report
15 The experiments with GATree were run on a Pentium IV 2.8 GHz machine with the Windows
XP operating system. The reported times are as output by the application itself.
66