Foundations of Data Science

We will show that the following algorithm satisfies these three axioms.
Balanced k-means algorithm
Among all partitions of the input set of n points into k sets, each of size n/k, return
the one that minimizes the sum of squared distances between all pairs of points in
the same cluster.
Theorem 8.15 The balanced k-means algorithm satisfies the consistency condition, scale
invariance, and the richness property.
Proof: Scale invariance is obvious. Richness is also easy to see. Just place n/k points of
S to coincide with each point of K. To prove consistency, define the cost of a cluster T
to be the sum of squared distances of all pairs of points in T .
Suppose S 1 , S 2 , . . . , S k is an optimal clustering of S according to the balanced k-
means algorithm. Move a point x ∈ S 1 to z so that its distance to each point in S 1 is
non increasing and its distance to each point in S 2 , S 3 , . . . , S k is non decreasing. Suppose
T 1 , T 2 , . . . , T k is an optimal clustering after the move. Without loss of generality assume
z ∈ T 1 . Define ˜T 1 = (T 1 \ {z}) ∪ {x} and ˜S 1 = (S 1 \ {x}) ∪ {z}. Note that ˜T 1 , T 2 , . . . , T k
is a clustering before the move, although not necessarily an optimal clustering. Thus
( )
cost ˜T1 + cost (T 2 ) + · · · + cost (T k ) ≥ cost (S 1 ) + cost (S 2 ) + · · · + cost (S k ) .
( ) ( )
If cost (T 1 ) − cost ˜T1 ≥ cost ˜S1 − cost (S 1 ) then
( )
cost (T 1 ) + cost (T 2 ) + · · · + cost (T k ) ≥ cost ˜S1 + cost (S 2 ) + · · · + cost (S k ) .
Since T 1 , T 2 , . . . , T k is an optimal clustering after the move, so also must be ˜S 1 , S 2 , . . . , S k
proving the theorem.
( ) ( )
It remains to show that cost (T 1 ) − cos t ˜T1 ≥ cost ˜S1 − cost (S 1 ). Let u and v
stand for elements other than x and z in S 1 and T 1 . The terms |u − v| 2 are common to T 1
and ˜T 1 on the left hand side and cancel out. So too on the right hand side. So we need
only prove ∑
(|z − u| 2 − |x − u| 2 ) ≥ ∑ (|z − u| 2 − |x − u| 2 ).
u∈T 1 u∈S 1
For u ∈ S 1 ∩ T 1 , the terms appear on both sides, and we may cancel them, so we are left
to prove ∑
(|z − u| 2 − |x − u| 2 ) ≥
∑
(|z − u| 2 − |x − u| 2 )
u∈T 1 \S 1 u∈S 1 \T 1
which is true because by the movement of x to z, each term on the left hand side is non
negative and each term on the right hand side is non positive.
294