anytime algorithms for learning anytime classifiers saher ... - Technion

Technion - Computer Science Department - Ph.D. Thesis PHD-2008-12 - 2008
Maimon, and Eberbach (2001) introduced an interruptible anytime algorithm for
feature selection. Their proposed method performs a hill-climbing forward search
in the space of feature subsets where the attributes are added to the selected
subset one at a time. constructed from the previously selected attributes. The
algorithm can be stopped at any moment, returning the set of currently selected
features.
All these methods are orthogonal to our induction methods and can complement
them. An interesting research direction is to determine resource distribution
between the various learning stages.
Costs are also involved in the learning phase, during example acquisition and
during model learning. The problem of budgeted learning has been studied by
Lizotte et al. (2003). There is a cost associated with obtaining each attribute
value of a training example, and the task is to determine what attributes to test
given a budget.
A related problem is active feature-value acquisition. In this setup one tries to
reduce the cost of improving accuracy by identifying highly informative instances.
Melville et al. (2004) introduced an approach in which instances are selected for
acquisition based on the accuracy of the current model and the model’s confidence
in the prediction.
An extension to the work on optimal fixed depth trees (Greiner et al., 2002)
is when tests have costs, both during learning and classification. Kapoor and
Greiner (2005) presented a budgeted learning framework that operates under
strict budget for feature-value acquisition during learning and produces a classifier
with a limited classification budget. Provost, Melville, and Saar-Tsechansky
(2007) discussed the challenges electronic commerce environments bring to data
acquisition during learning and classification. They discussed several settings and
presented a unified framework to integrate acquisition of different types, with any
cost structure and any predictive modeling objective.
(Madani, Lizotte, & Greiner, 2004) studied the setup of budgeted active model
selection. In this setup the learner can use a fixed budget of model probes to
identify which of a given set of possible models has the highest expected accuracy.
In each probe a model is evaluated on a random indistinguishable instance. The
goal is a policy that sequentially determines which model to probe next, based on
the information observed so far. Madani et al. formalized the task and showed
that it is NP-hard in general. They also studied several algorithms for budgeted
active model selection and compared them empirically.
133