Building Machine Learning Systems with Python - Richert, Coelho

Chapter 3
Flat clustering divides the posts into a set of clusters without relating the clusters to
each other. The goal is simply to come up with a partitioning such that all posts in
one cluster are most similar to each other while being dissimilar from the posts in all
other clusters. Many flat clustering algorithms require the number of clusters to be
specified up front.
In hierarchical clustering, the number of clusters does not have to be specified.
Instead, the hierarchical clustering creates a hierarchy of clusters. While similar posts
are grouped into one cluster, similar clusters are again grouped into one uber-cluster.
This is done recursively, until only one cluster is left, which contains everything. In
this hierarchy, one can then choose the desired number of clusters. However, this
comes at the cost of lower efficiency.
Scikit provides a wide range of clustering approaches in the package sklearn.
cluster. You can get a quick overview of the advantages and drawbacks of each
of them at http://scikit-learn.org/dev/modules/clustering.html.
In the following section, we will use the flat clustering method, KMeans, and play
a bit with the desired number of clusters.
KMeans
KMeans is the most widely used flat clustering algorithm. After it is initialized with
the desired number of clusters, num_clusters, it maintains that number of so-called
cluster centroids. Initially, it would pick any of the num_clusters posts and set the
centroids to their feature vector. Then it would go through all other posts and assign
them the nearest centroid as their current cluster. Then it will move each centroid
into the middle of all the vectors of that particular class. This changes, of course, the
cluster assignment. Some posts are now nearer to another cluster. So it will update
the assignments for those changed posts. This is done as long as the centroids move
a considerable amount. After some iterations, the movements will fall below a
threshold and we consider clustering to be converged.
Downloading the example code
You can download the example code files for all Packt books you
have purchased from your account at http://www.packtpub.
com. If you purchased this book elsewhere, you can visit http://
www.packtpub.com/support and register to have the files
e-mailed directly to you.
[ 63 ]