anytime algorithms for learning anytime classifiers saher ... - Technion

Technion - Computer Science Department - Ph.D. Thesis PHD-2008-12 - 2008
Test costs
a1
a2
a
a4
a5
a6
40
10
10
10
10
10
T1 T2
a1
40 60
- +
a1
20 80
a2 a3
15 5 10 70
- + - +
Figure 5.9: An example of applying cost discounts when forming a repertoire for
interruptible classification. The numbers represent the number of examples that follow
each edge. Because the value of a1 is required at the root of T1, any subsequent tree
can obtain this value at no cost. The probability for testing a2 is 50% in T2. When
inducing T3, the attribute a2 has already been measured with a probability of 50%.
Hence, we discount the cost of a2 by 50% ($5 instead of $10). Similarly, the cost of
a3 is discounted by 80% ($2 instead of $10).
previous trees.
Consider, for example, the trees in Figure 5.9. The probability to measure a1
in T1 is 100%. Therefore, when building subsequent trees, the cost of a1 would
be zero. The probability for testing a2 is in T2 20%. Hence, when inducing T3,
we discount the cost of a2 by 20% ($8 instead of $10). Similarly, the cost of a3 is
discounted by 80% ($2 instead of $10).
Because the trees may be strongly correlated, we cannot simply calculate this
probability independently for each tree. For example, if T3 in the aforementioned
example tests a2 for 70% of the examples, we would like to know for how many
of these examples a2 has been tested also in T2. Therefore, we traverse the
previous trees with each of the training examples and mark the attributes that
are tested at least once. For efficiency, the matrix that represents which tests were
administered for which case is built incrementally and updated after building each
new tree.
We refer to this method as discount repertoire. The repertoire is formed using
the same method in Figure 5.5 with a single change: before building each tree,
cost discounts are applied; the discounts are based on the trees already in the
repertoire. Figure 5.10 formalizes the procedure for updating test costs. During
classification we iterate over the trees until interrupted, as described in Figure
5.8.
5.4 Empirical Evaluation
A variety of experiments were conducted to test the performance and behavior of
TATA in 3 different setups: pre-contract, contract, and interruptible. In Chapter
110