Data Mining: Practical Machine Learning Tools and ... - LIDeCC

6.5 NUMERIC PREDICTION 243eralization is. Salzberg (1991) suggested that generalization with nested exemplarscan achieve a high degree of classification of accuracy on a variety of differentproblems, a conclusion disputed by Wettschereck and Dietterich (1995),who argued that these results were fortuitous and did not hold in otherdomains. Martin (1995) explored the idea that it is not the generalization butthe overgeneralization that occurs when hyperrectangles nest or overlap that isresponsible for poor performance and demonstrated that if nesting and overlappingare avoided excellent results are achieved in a large number of domains.The generalized distance function based on transformations is described byCleary and Trigg (1995).Exemplar generalization is a rare example of a learning strategy in which thesearch proceeds from specific to general rather than from general to specific asin the case of tree or rule induction. There is no particular reason why specificto-generalsearching should necessarily be handicapped by forcing the examplesto be considered in a strictly incremental fashion, and batch-orientedapproaches exist that generate rules using a basic instance-based approach.Moreover, it seems that the idea of producing conservative generalizations andcoping with instances that are not covered by choosing the “closest” generalizationis an excellent one that will eventually be extended to ordinary tree andrule inducers.6.5 Numeric predictionTrees that are used for numeric prediction are just like ordinary decisiontrees except that at each leaf they store either a class value that represents theaverage value of instances that reach the leaf, in which case the tree is called aregression tree, or a linear regression model that predicts the class value ofinstances that reach the leaf, in which case it is called a model tree. In whatfollows we will describe model trees because regression trees are really a specialcase.Regression and model trees are constructed by first using a decision treeinduction algorithm to build an initial tree. However, whereas most decisiontree algorithms choose the splitting attribute to maximize the information gain,it is appropriate for numeric prediction to instead minimize the intrasubsetvariation in the class values down each branch. Once the basic tree has beenformed, consideration is given to pruning the tree back from each leaf, just aswith ordinary decision trees. The only difference between regression tree andmodel tree induction is that for the latter, each node is replaced by a regressionplane instead of a constant value. The attributes that serve to define that regressionare precisely those that participate in decisions in the subtree that will bepruned, that is, in nodes underneath the current one.