anytime algorithms for learning anytime classifiers saher ... - Technion

Technion - Computer Science Department - Ph.D. Thesis PHD-2008-12 - 2008
Chapter 7
Conclusions
The application of machine learning techniques to real world problems involves
several types of costs and constraints. In this thesis we proposed a novel framework
for operating under different constraints during learning and classification.
Our framework allows computation speed during learning to be traded for better
predictive models.
In Chapter 2 we described a variety of possible scenarios of resource allocation
and consumption. In contract learning, the learning resources are preallocated
and provided to the learner. In interruptible learning, the learner should utilize
extra time until interrupted and queried for a solution. Therefore, an interruptible
learner should be ready to return a valid classifier at any time.
Using a classifier for predicting the labels of new cases carries two costs: the
cost of the tests the model requires to administer, and the cost of the predictive
errors the model makes. In cost-insensitive classification, tests do not have costs
and the penalty of wrong classification is uniform, no matter what the error type
is. Therefore, the objective of a learning algorithm in this case is to produce
comprehensible and accurate models. In cost-sensitive classification, on the other
hand, the goal is to minimize the total cost, i.e., the sum of testing costs and
misclassification costs. Many real-world applications, however, limit the costs
of the tests a model can require. Therefore, another interesting objective we
studied is to minimize misclassification costs when testing costs are bounded. The
bound on testing costs may be predetermined and provided to the learner (precontract
classification), known to the classifier but not to the learner (contract
classification), or unknown; the classifier is then expected to exploit resources
until interrupted (interruptible classification). In this thesis we handled all these
scenarios.
In Chapter 3 we presented a general framework for contract anytime induction
of decision trees. The major limitation of top-down greedy tree learners is
their inability to recover from wrong split decisions that might arise when local
135