anytime algorithms for learning anytime classifiers saher ... - Technion

Technion - Computer Science Department - Ph.D. Thesis PHD-2008-12 - 2008
significant with α = 0.05. Note that binary splits come at a price: the number
of candidate splits increases and the runtime becomes significantly longer. When
allocated the same time budget, LSID3-p can afford larger samples than BLSID3p.
The advantage of the latter, however, is kept.
The second task is a variant on XOR-2, where there are 3 attributes a1, a2, a3,
each of which can take one of the values A, C, G, T. The target concept is a ∗ 1 ⊕a∗ 2 .
The values of a ∗ i are obtained from ai by mapping each A and C to 0 and each
G and T to 1. The dataset, referred to as Categorial XOR-2, consists of 16
randomly drawn examples. With multiway splits, both LSID3-p and C4.5 could
not learn Categorial XOR-2 and their accuracy was about 50%. C4.5 failed also
with binary splits. BLSID3-p, on the contrary, was 92% accurate.
We also examined the bias of LSID3 toward binary attributes. For this purpose
we used the example described in Section 3.5.2. An artificial dataset with
all possible values was created. LSID3 with multiway and with binary splits were
run 10000 times. Although a4 is as important to the class as a1 and a2, LSID3
with multiway splits never chose it at the root. LSID3 with binary splits, however,
split the root on a4 35% of the time. These results were similar to a1 (33%)
and a2 (32%). They indicate that forcing binary splits removes the LSID3 bias.
3.7.3 Anytime Behavior of the Contract Algorithms
Both LSID3 and ID3-k make use of additional resources for generating better
trees. However, in order to serve as good anytime algorithms, the quality of their
output should improve with the increase in their allocated resources. For a typical
anytime algorithm, this improvement is greater at the beginning and diminishes
over time. To test the anytime behavior of LSID3 and ID3-k, we invoked them
with successive values of r and k respectively. In the first set of experiments we
focused on domains with nominal attributes, while in the second set we tested
the anytime behavior for domains with continuous attributes.
Nominal attributes
Figures 3.23, 3.24 and 3.25 show the average results over 10 runs of 10-fold crossvalidation
experiments for the Multiplex-XOR, XOR-10 and Tic-tac-toe datasets
respectively. The x-axis represents the run time in seconds. 13 ID3 and C4.5,
which are constant time algorithms, terminate quickly and do not improve with
time. Since ID3-k with k = 1 and LSID3 with r = 0 are defined to be identical
to ID3, the point at which ID3 yields a result serves also as the starting point of
these anytime algorithms.
13 The algorithms were implemented in C++, compiled with GCC, and run on Macintosh G5
2.5 GHz.
54