Data Mining: Practical Machine Learning Tools and ... - LIDeCC

196 CHAPTER 6 | IMPLEMENTATIONS: REAL MACHINE LEARNING SCHEMESComplexity of decision tree inductionNow that we have learned how to accomplish the pruning operations, we havefinally covered all the central aspects of decision tree induction. Let’s take stockand consider the computational complexity of inducing decision trees. We willuse the standard order notation: O(n) stands for a quantity that grows at mostlinearly with n, O(n 2 ) grows at most quadratically with n, and so on.Suppose that the training data contains n instances and m attributes. We needto make some assumption about the size of the tree, and we will assume that itsdepth is on the order of log n, that is, O(log n). This is the standard rate ofgrowth of a tree with n leaves, provided that it remains “bushy” and doesn’tdegenerate into a few very long, stringy branches. Note that we are tacitly assumwageincrease first year≤ 2.5 > 2.5working hoursper week≤ 36> 361 bad1 goodhealth plan contributionnone half full4 bad2 good1 bad1 good4 bad2 goodFigure 6.2 Pruning the labor negotiations decision tree.the error estimate for the working hours node, so the subtree is pruned away andreplaced by a leaf node.The estimated error figures obtained in these examples should be taken witha grain of salt because the estimate is only a heuristic one and is based on anumber of shaky assumptions: the use of the upper confidence limit; theassumption of a normal distribution; and the fact that statistics from the trainingset are used. However, the qualitative behavior of the error formula is correctand the method seems to work reasonably well in practice. If necessary, theunderlying confidence level, which we have taken to be 25%, can be tweaked toproduce more satisfactory results.