anytime algorithms for learning anytime classifiers saher ... - Technion

Technion - Computer Science Department - Ph.D. Thesis PHD-2008-12 - 2008
Average Size
Average Accuracy
40
38
36
34
32
30
28
26
24
0 50 100 150 200 250 300
72
70
68
66
Time [sec]
ID3
C4.5
IIDT(1)
IIDT(0.1)
64
ID3
C4.5
IIDT(1)
62
IIDT(0.1)
0 50 100 150 200 250 300 350
Time [sec]
Figure 3.28: Anytime behavior of IIDT on the Glass dataset
3.30 show the anytime performance of IIDT in terms of tree size and accuracy for
the Glass, XOR-10, and Tic-tac-toe datasets. Each graph represents an average
of 100 runs (for the 10 ×10 cross-validation). Unlike the graphs given in the previous
section, these are interruptible anytime graphs, i.e., for each point, the y
coordinate reflects the performance if the algorithm was interrupted at the associated
x coordinate. In the contract algorithm graphs, however, each point reflects
the performance if the algorithm was initially allocated the time represented by
the x coordinate.
In all cases, the two anytime versions indeed exploit the additional resources
and produce both smaller and more accurate trees. Since our algorithm replaces a
subtree only if the new one is smaller, all size graphs decrease monotonically. The
most interesting anytime behavior is for the difficult XOR-10 problem. There,
the tree size decreases from 4000 leaves to almost the optimal size (1024), and the
accuracy increases from 50% (which is the accuracy achieved by ID3 and C4.5) to
almost 100%. The shape of the graphs is typical to those of anytime algorithms
60