Learning Data Mining with Python

Clustering News Articles
You can then perform your own second-level clustering on the result, or use it for
classification if you have the target values. A possible workflow for this would be to
perform some feature selection using the supervised data, use clustering to reduce
the number of features to a more manageable number, and then use the results in a
classification algorithm such as SVMs.
Clustering ensembles
In Chapter 3, Predicting Sports Winners with Decision Trees, we looked at a classification
ensemble using the random forest algorithm, which is an ensemble of many lowquality,
tree-based classifiers. Ensembling can also be performed using clustering
algorithms. One of the key reasons for doing this is to smooth the results from many
runs of an algorithm. As we saw before, the results from running k-means are varied,
depending on the selection of the initial centroids. Variation can be reduced by
running the algorithm many times and then combining the results.
Ensembling also reduces the effects of choosing parameters on the final result.
Most clustering algorithms are quite sensitive to the parameter values chosen for
the algorithm. Choosing slightly different parameters results in different clusters.
Evidence accumulation
As a basic ensemble, we can first cluster the data many times and record the labels
from each run. We then record how many times each pair of samples was clustered
together in a new matrix. This is the essence of the Evidence Accumulation
Clustering (EAC) algorithm.
EAC has two major steps. The first step is to cluster the data many times using
a lower-level clustering algorithm such as k-means and record the frequency
that samples were in the same cluster, in each iteration. This is stored in a
coassociation matrix. The second step is to perform a cluster analysis on the
resulting coassociation matrix, which is performed using another type of
clustering algorithm called hierarchical clustering. This has an interesting property,
as it is mathematically the same as finding a tree that links all the nodes together
and removing weak links.
We can create a coassociation matrix from an array of labels by iterating over
each of the labels and recording where two samples have the same label. We use
SciPy's csr_matrix, which is a type of sparse matrix:
from scipy.sparse import csr_matrix
[ 230 ]