anytime algorithms for learning anytime classifiers saher ... - Technion

Technion - Computer Science Department - Ph.D. Thesis PHD-2008-12 - 2008
Average size
Average Accuracy
4000
3500
3000
2500
2000
1500
ID3
C4.5
IIDT(1)
IIDT(0.1)
1000
0 100 200 300 400 500 600
100
95
90
85
80
75
70
65
60
55
50
45
Time [sec]
0 100 200 300 400 500 600
Time [sec]
ID3
C4.5
IIDT(1)
IIDT(0.1)
Figure 3.29: Anytime behavior of IIDT on the 10-XOR dataset
with diminishing returns. The improvement in the accuracy of IIDT (at the
latest point it was measured) over ID3 and C4.5 was found by t-test (α = 0.05)
to be significant for the Glass and XOR-10 datasets. The performance of IIDT
on Tic-tac-toe slightly degrades over time. We believe that similarly to LSID3,
IIDT can perform much better if binary splits are used.
The difference in performance of the two anytime algorithms is interesting.
IIDT(0.1), with the lower granularity parameter, indeed produces smoother anytime
graphs (with lower volatility), which allows for better control and better predictability
of return. Moreover, in large portions of the time axis, the IIDT(0.1)
graph shows better performance than that of IIDT(1). This is due to more sophisticated
node selection in the former. Recall that g = 1 means that the algorithm
always selects the entire tree for improvement.
The smoothness of the IIDT(0.1) graphs is somehow misleading because it
represents an average of 100 runs, with each step taking place at a different time
(this is in contrast to the graph for IIDT(1), where the steps are at roughly the
61