Data Mining: Practical Machine Learning Tools and ... - LIDeCC

7.7 FURTHER READING 343Domingos (1997) describes how to derive a single interpretable model froman ensemble using artificial training examples. Bayesian option trees were introducedby Buntine (1992), and majority voting was incorporated into optiontrees by Kohavi and Kunz (1997). Freund and Mason (1999) introduced alternatingdecision trees; experiments with multiclass alternating decision treeswere reported by Holmes et al. (2002). Landwehr et al. (2003) developed logisticmodel trees using the LogitBoost algorithm.Stacked generalization originated with Wolpert (1992), who presented theidea in the neural network literature, and was applied to numeric prediction byBreiman (1996a). Ting and Witten (1997a) compared different level-1 modelsempirically and found that a simple linear model performs best; they alsodemonstrated the advantage of using probabilities as level-1 data. A combinationof stacking and bagging has also been investigated (Ting and Witten1997b).The idea of using error-correcting output codes for classification gained wideacceptance after a paper by Dietterich and Bakiri (1995); Ricci and Aha (1998)showed how to apply such codes to nearest-neighbor classifiers.Blum and Mitchell (1998) pioneered the use of co-training and developed atheoretical model for the use of labeled and unlabeled data from different independentperspectives. Nigam and Ghani (2000) analyzed the effectiveness andapplicability of co-training, relating it to the traditional use of standard EM tofill in missing values. They also introduced the co-EM algorithm. Nigam et al.(2000) thoroughly explored how the EM clustering algorithm can use unlabeleddata to improve an initial classifier built by Naïve Bayes, as reported in theClustering for classification section. Up to this point, co-training and co-EMwere applied mainly to small two-class problems; Ghani (2002) used errorcorrectingoutput codes to address multiclass situations with many classes.Brefeld and Scheffer (2004) extended co-EM to use a support vector machinerather than Naïve Bayes. Seeger (2001) casts some doubt on whether these newalgorithms really do have anything to offer over traditional ones, properly used.