Data Mining: Practical Machine Learning Tools and ... - LIDeCC

254 CHAPTER 6 | IMPLEMENTATIONS: REAL MACHINE LEARNING SCHEMESweighted learning, primarily in the context of regression problems. Frank et al.(2003) evaluated the use of locally weighted learning in conjunction with NaïveBayes.6.6 ClusteringIn Section 4.8 we examined the k-means clustering algorithm in which k initialpoints are chosen to represent initial cluster centers, all data points are assignedto the nearest one, the mean value of the points in each cluster is computed toform its new cluster center, and iteration continues until there are no changesin the clusters. This procedure only works when the number of clusters is knownin advance, and this section begins by describing what you can do if it is not.Next we examine two techniques that do not partition instances into disjointclusters as k-means does. The first is an incremental clustering method that wasdeveloped in the late 1980s and embodied in a pair of systems called Cobweb(for nominal attributes) and Classit (for numeric attributes). Both come up witha hierarchical grouping of instances and use a measure of cluster “quality” calledcategory utility. The second is a statistical clustering method based on a mixturemodel of different probability distributions, one for each cluster. It assignsinstances to classes probabilistically, not deterministically. We explain the basictechnique and sketch the working of a comprehensive clustering scheme calledAutoClass.Choosing the number of clustersSuppose you are using k-means but do not know the number of clusters inadvance. One solution is to try out different possibilities and see which is best—that is, which one minimizes the total squared distance of all points to theircluster center. A simple strategy is to start from a given minimum, perhapsk = 1, and work up to a small fixed maximum, using cross-validation to find thebest value. Because k-means is slow, and cross-validation makes it even slower,it will probably not be feasible to try many possible values for k. Note that onthe training data the “best” clustering according to the total squared distancecriterion will always be to choose as many clusters as there are data points! Topenalize solutions with many clusters you have to apply something like the MDLcriterion of Section 5.10, or use cross-validation.Another possibility is to begin by finding a few clusters and determiningwhether it is worth splitting them. You could choose k = 2, perform k-meansclustering until it terminates, and then consider splitting each cluster. Computationtime will be reduced considerably if the initial two-way clusteringis considered irrevocable and splitting is investigated for each component