Foundations of Data Science

probability is q (where, q < p). 32 We assume the events that person i knows person j are
independent across all i and j.
Specifically, we are given an n × n data matrix A, where, a ij = 1 if and only if i and
j know each other. We assume the a ij are independent random variables, and use a i to
denote the i th row of A. It is useful to think of A as the adjacency matrix of a graph, such
as the friendship network in Facebook. We will also think of the rows a i as data points.
The clustering problem is to classify the data points into the communities they belong to.
In practice, the graph is fairly sparse, i.e., p and q are small, namely, O(1/n) or O(ln n/n).
Consider the simple case of two communities with n/2 people in each and with
p = α n ; q = β n
, where, α, β ∈ O(ln n).
Let u and v be the centroids of the data points in community one and community two
respectively; so, u i ≈ p for i ∈ C 1 and u j ≈ q for j ∈ C 2 and v i ≈ q for i ∈ C 1 and v j ≈ p
for j ∈ C 2 . We have
|u − v| 2 ≈
n∑
(u j − v j ) 2 =
j=1
(α − β)2
n 2 n =
(α − β)2
.
n
Inter-centroid distance ≈ α − β √ n
. (8.1)
On the other hand, the distance between a data point and its cluster centroid is much
greater:
For i ∈ C 1 , E ( |a i − u| 2) n∑
= E((a ij − u j ) 2 ) = n [p(1 − p) + q(1 − q)] ∈ Ω(α + β).
2
j=1
We see that if α, β ∈ O(ln n), then the ratio
(√ )
distance between cluster centroids
ln n
distance of data point to its cluster centroid ∈ O √ . n
So the centroid of a cluster is much farther from data points in its own cluster than it is
to the centroid of the other cluster.
Now, consider the k-median objective function. Suppose we wrongly classify a point
in C 1 as belonging to C 2 . The extra cost we incur is at most the distance between
centroids which is only O( √ ln n/ √ n) times the k-median cost of the data point. So just
by examining the cost, we cannot rule out a ε-approximate k-median clustering from
misclassifying all points. A similar argument can also be made about k-means objective
function.
32 More generally, for each pair of communities a and b, there could be a probability p ab that a person
from community a knows a person from community b. But for the discussion here, we take p aa = p for
all a and p ab = q, for all a ≠ b.
277