Learning Data Mining with Python

Chapter 10
We use clustering techniques when we don't have a clear set of target classes for
our data. In that sense, clustering algorithms have little direction in their learning.
They learn according to some function, regardless of the underlying meaning of the
data. For this reason, it is critical to choose good features. In supervised learning,
if you choose poor features, the learning algorithm can choose to not use those
features. For instance, support vector machines will give little weight to features
that aren't useful in classification. However, with clustering, all features are used in
the final result—even if those features don't provide us with the answer we were
looking for.
When performing cluster analysis on real-world data, it is always a good idea to
have a sense of what sorts of features will work for your scenario. In this chapter, we
will use the bag-of-words model. We are looking for topic-based groups, so we will use
topic-based features to model the documents. We know those features work because
of the work others have done in supervised versions of our problem. In contrast, if we
were to perform an authorship-based clustering, we would use features such as those
found in the Chapter 9, Authorship Attribution experiment.
The k-means algorithm
The k-means clustering algorithm finds centroids that best represent the data using
an iterative process. The algorithm starts with a predefined set of centroids, which
are normally data points taken from the training data. The k in k-means is the
number of centroids to look for and how many clusters the algorithm will find.
For instance, setting k to 3 will find three clusters in the dataset.
There are two phases to the k-means: assignment and updating.
In the assignment step, we set a label to every sample in the dataset linking it to the
nearest centroid. For each sample nearest to centroid 1, we assign the label 1. For each
sample nearest to centroid 2, we assign a label 2 and so on for each of the k centroids.
These labels form the clusters, so we say that each data point with the label 1 is in
cluster 1 (at this time only, as assignments can change as the algorithm runs).
In the updating step, we take each of the clusters and compute the centroid, which
is the mean of all of the samples in that cluster.
[ 223 ]