Foundations of Data Science

8.13 Exercises
Exercise 8.1 Construct examples where using distances instead of distance squared gives
bad results for Gaussian densities. For example, pick samples from two 1-dimensional
unit variance Gaussians, with their centers 10 units apart. Cluster these samples by trial
and error into two clusters, first according to k-means and then according to the k-median
criteria. The k-means clustering should essentially yield the centers of the Gaussians as
cluster centers. What cluster centers do you get when you use the k-median criterion?
Exercise 8.2 Let v = (1, 3). What is the L 1 norm of v? The L 2 norm? The square of
the L 1 norm?
Exercise 8.3 Show that in 1-dimension, the center of a cluster that minimizes the sum
of distances of data points to the center is in general not unique. Suppose we now require
the center also to be a data point; then show that it is the median element (not the mean).
Further in 1-dimension, show that if the center minimizes the sum of squared distances
to the data points, then it is unique.
Exercise 8.4 Construct a block diagonal matrix A with three blocks of size 50. Each
matrix element in a block has value p = 0.7 and each matrix element not in a block has
value q = 0.3. Generate a 150 × 150 matrix B of random numbers in the range [0,1]. If
b ij ≥ a ij replace a ij with the value one. Otherwise replace a ij with value zero. The rows
of A have three natural clusters. Generate a random permutation and use it to permute
the rows and columns of the matrix A so that the rows and columns of each cluster are
randomly distributed.
1. Apply the k-mean algorithm to A with k = 3. Do you find the correct clusters?
2. Apply the k-means algorithm to A for 1 ≤ k ≤ 10. Plot the value of the sum of
squares to the cluster centers versus k. Was three the correct value for k?
Exercise 8.5 Let M be a k × k matrix whose elements are numbers in the range [0,1].
A matrix entry close to one indicates that the row and column of the entry correspond to
closely related items and an entry close to zero indicates unrelated entities. Develop an
algorithm to match each row with a closely related column where a column can be matched
with only one row.
Exercise 8.6 The simple greedy algorithm of Section 8.3 assumes that we know the clustering
radius r. Suppose we do not. Describe how we might arrive at the correct r?
Exercise 8.7 For the k-median problem, show that there is at most a factor of two ratio
between the optimal value when we either require all cluster centers to be data points or
allow arbitrary points to be centers.
Exercise 8.8 For the k-means problem, show that there is at most a factor of four ratio
between the optimal value when we either require all cluster centers to be data points or
allow arbitrary points to be centers.
295