Foundations of Data Science

a clustering that is ɛ-close to C T . That is, we can perform as well as if we had a generic
1.1-factor approximation algorithm, even though achieving a 1.1-factor approximation is
NP-hard in general. Results known for the k-means objective are somewhat weaker, finding
a clustering that is O(ɛ)-close to C T . Here, we show this for the k-median objective
where the analysis is cleanest. We make a few additional assumptions in order to focus
on the main idea. In the following one should think of ɛ as o( c−1 ). The results described
k
here apply to data in any metric space, it need not be R d . 31
For simplicity and ease of notation assume that C T = C ∗ ; that is, the target clustering
is also the optimum for the objective. For a given data point a i , define its weight w(a i ) to
be its distance to the center of its cluster in C ∗ . Notice that for the k-median objective,
we have Φ(C ∗ ) = ∑ n
i=1 w(a i). Define w avg = Φ(C ∗ )/n to be the average weight of the
points in A. Finally, define w 2 (a i ) to be the distance of a i to its second-closest center in
C ∗ . We now begin with a useful lemma.
Lemma 8.4 Assume dataset A satisfies (c, ɛ) approximation-stability with respect to the
k-median objective, each cluster in C T has size at least 2ɛn, and C T = C ∗ . Then,
1. Fewer than ɛn points a i have w 2 (a i ) − w(a i ) ≤ (c − 1)w avg /ɛ.
2. At most 5ɛn/(c − 1) points a i have w(a i ) ≥ (c − 1)w avg /(5ɛ).
Proof: For part (1), suppose that ɛn points a i have w 2 (a i ) − w(a i ) ≤ (c − 1)w avg /ɛ.
Consider modifying C T to a new clustering C ′ by moving each of these points a i into
the cluster containing its second-closest center. By assumption, the k-means cost of the
clustering has increased by at most ɛn(c − 1)w avg /ɛ = (c − 1)Φ(C ∗ ). This means that
Φ(C ′ ) ≤ c · Φ(C ∗ ). However, dist(C ′ , C T ) = ɛ because (a) we moved ɛn points to different
clusters, and (b) each cluster in C T has size at least 2ɛn so the optimal permutation σ in the
definition of dist remains the identity. So, this contradicts approximation stability. Part
(2) follows from the definition of “average”; if it did not hold then ∑ n
i=1 w(a i) > nw avg ,
a contradiction.
A datapoint a i is bad if it satisfies either item (1) or (2) of Lemma 8.4 and good if it
satisfies neither one. So, there are at most b = ɛn + 5ɛn bad points and the rest are good.
c−1
Define “critical distance” d crit = (c−1)wavg . So, Lemma 8.4 implies that the good points
5ɛ
have distance at most d crit to the center of their own cluster in C ∗ and distance at least
5d crit to the center of any other cluster in C ∗ .
This suggests the following algorithm. Suppose we create a graph G with the points
a i as vertices, and edges between any two points a i and a j with d(a i , a j ) < 2d crit . Notice
31 An example of a data set satisfying (2, 0.2/k) approximation stability would be k clusters of n/k
points each, where in each cluster, 90% of the points are within distance 1 of the cluster center (call this
the “core” of the cluster), with the other 10% arbitrary, and all cluster cores are at distance at least 10k
apart.
273