Modeling and Multivariate Methods - SAS

468 Clustering Data Chapter 18
Hierarchical Clustering
Ward’s In Ward’s minimum variance method, the distance between two clusters is the ANOVA sum of
squares between the two clusters added up over all the variables. At each generation, the within-cluster
sum of squares is minimized over all partitions obtainable by merging two clusters from the previous
generation. The sums of squares are easier to interpret when they are divided by the total sum of squares
to give the proportions of variance (squared semipartial correlations).
Ward’s method joins clusters to maximize the likelihood at each level of the hierarchy under the
assumptions of multivariate normal mixtures, spherical covariance matrices, and equal sampling
probabilities.
Ward’s method tends to join clusters with a small number of observations and is strongly biased toward
producing clusters with approximately the same number of observations. It is also very sensitive to
outliers. See Milligan (1980).
Distance for Ward’s method is
2
x K – x L
D KL
= -------------------------
-------
1 1
+ ------
N K
N L
Single Linkage In single linkage the distance between two clusters is the minimum distance between an
observation in one cluster and an observation in the other cluster. Single linkage has many desirable
theoretical properties. See Jardine and Sibson (1976), Fisher and Van Ness (1971), and Hartigan (1981).
Single linkage has, however, fared poorly in Monte Carlo studies. See Milligan (1980). By imposing no
constraints on the shape of clusters, single linkage sacrifices performance in the recovery of compact
clusters in return for the ability to detect elongated and irregular clusters. Single linkage tends to chop
off the tails of distributions before separating the main clusters. See Hartigan (1981). Single linkage was
originated by Florek et al. (1951a, 1951b) and later reinvented by McQuitty (1957) and Sneath (1957).
Distance for the single linkage cluster method is
D KL
= min i ∈ CK
min j ∈
CL
d ( x i
, x j
)
Complete Linkage In complete linkage, the distance between two clusters is the maximum distance
between an observation in one cluster and an observation in the other cluster. Complete linkage is
strongly biased toward producing clusters with approximately equal diameters and can be severely
distorted by moderate outliers. See Milligan (1980).
Distance for the Complete linkage cluster method is
D KL
= max i ∈ CK
max j ∈
CL
d ( x i
, x j
)
Fast Ward is a way of applying Ward's method more quickly for large numbers of rows. It is used
automatically whenever there are more than 2000 rows.