anytime algorithms for learning anytime classifiers saher ... - Technion

Technion - Computer Science Department - Ph.D. Thesis PHD-2008-12 - 2008
Contents
Abstract 1
1 Introduction 5
1.1 Proposed Solution . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.2 Major Contributions . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.3 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2 Resource-bounded Learning and Classification 13
3 Contract Anytime Learning of Accurate Trees 19
3.1 Top-down Induction of Decision Trees . . . . . . . . . . . . . . . 19
3.2 Limitation of Greedy Learners . . . . . . . . . . . . . . . . . . . 20
3.3 Tree-size as a Preference Bias . . . . . . . . . . . . . . . . . . . . 22
3.4 Fixed-depth Lookahead . . . . . . . . . . . . . . . . . . . . . . . 24
3.5 A Contract Algorithm for Learning Decision trees . . . . . . . . . 25
3.5.1 Evaluating Continuous Attributes by Sampling . . . . . . 30
3.5.2 Multiway Vs. Binary Splits . . . . . . . . . . . . . . . . . 32
3.5.3 Pruning the LSID3 Trees . . . . . . . . . . . . . . . . . . 32
3.5.4 Mapping Contract Time to Sample Size . . . . . . . . . . 33
3.6 Interruptible Induction of Decision Trees . . . . . . . . . . . . . . 34
3.6.1 Interruptible Learning by Sequencing . . . . . . . . . . . . 35
3.6.2 Iterative Improvement of Decision Trees . . . . . . . . . . 36
3.6.3 Incremental Learning: When New Examples Arrive . . . . 42
3.7 Empirical Evaluation . . . . . . . . . . . . . . . . . . . . . . . . 45
3.7.1 Experimental Methodology . . . . . . . . . . . . . . . . . 45
3.7.2 Fixed Time Comparison . . . . . . . . . . . . . . . . . . . 46
3.7.3 Anytime Behavior of the Contract Algorithms . . . . . . . 54
3.7.4 Anytime behavior of IIDT . . . . . . . . . . . . . . . . . . 59
3.7.5 IIDT as Incremental Learner . . . . . . . . . . . . . . . . 62
3.7.6 Comparison with Modern Decision Tree Learners . . . . . 63