New Statistical Algorithms for the Analysis of Mass - FU Berlin, FB MI ...

84 CHAPTER 4. (BIO-)MEDICAL APPLICATIONS
(d is the dimension of a cluster parameter space) such that members of a
particular cluster are close to this cluster and far away from other cluster. It
is obvious that such methods hinge on the notation of distance. Let
d(x, θi) : X × Ω → [0, ∞) , (4.3.3)
be a functional describing the distance from the observation x to the cluster i.
For a given cluster distance functional (4.3.3), under data clustering we will
understand the problem of a function Γ(x) = (γ1(x), . . . , γK(x)) called the
cluster affiliation (or the cluster weights) for a datum x together with cluster
parameters Θ = (θ1, . . . , θK) which minimize the cluster scoring functional
L(Θ, Γ) =
|X|
�
K�
t=1 i=1
subject to the constraints on Γ(x):
γi(xt) · d (xt, θi) → min , (4.3.4)
Γ(x),Θ
K�
γi(x) = 1, ∀x ∈ X (4.3.5)
i=1
γi(x) ≥ 0, ∀x ∈ X, i = 1, . . . , K. (4.3.6)
As we will see below choice of the cluster distance functional d (4.3.3)
will bias an algorithm towards finding different types of cluster structures (or
shapes) in the data. To illustrate this, we might choose d such that it will
favor clusters where each member is as close to the cluster center as possible.
We would expect these clusters to be compact and roughly spherical. On the
other hand, we could also define our d such that each cluster member is as
close to another cluster member - but not necessarily to all other members
or the cluster center. Clusters discovered by this approach need not to be
spherical or compact, but could have some sort of sausage shape.
A large number of different score functions can be used to measure the
quality of clustering and a wide range of algorithms has been developed to
search for an optimal (or at least good) partition. The exhaustive approach
would be to simply search through the space of possible assignments of n
points to K clusters to find that one that minimizes the score. The number
of possible allocations is approximately K n - thus, with n = 100 points and
K = 2 classes we would have to evaluate 2 100 ≈ 10 10 possible allocations.
Since this is - of course - not feasible, the next sections exemplarily introduce
concepts on how to practically optimize those score functions.
K-Means Clustering
One of the most popular clustering methods in multivariate data-analysis is
the so-called k-means algorithm (Bezdek, 1981; Höppner et al., 1999). The
affiliation to a certain cluster i is defined by the proximity of the observation
x ∈ X to the cluster center θi ∈ X. In this case the cluster distance functional
(4.3.3) takes the form of the square of the simple Euclidean distance between
the points in n dimensions: