Learning Data Mining with Python

Chapter 10
Our function definition takes a set of labels:
def create_coassociation_matrix(labels):
We then record the rows and columns of each match. We do these in a list. Sparse
matrices are commonly just sets of lists recording the positions of nonzero values,
and csr_matrix is an example of this type of sparse matrix:
rows = []
cols = []
We then iterate over each of the individual labels:
unique_labels = set(labels)
for label in unique_labels:
We look for all samples that have this label:
indices = np.where(labels == label)[0]
For each pair of samples with the preceding label, we record the position of both
samples in our list. The code is as follows:
for index1 in indices:
for index2 in indices:
rows.append(index1)
cols.append(index2)
Outside all loops, we then create the data, which is simply the value 1 for every time
two samples were listed together. We get the number of 1 to place by noting how
many matches we had in our labels set altogether. The code is as follows:
data = np.ones((len(rows),))
return csr_matrix((data, (rows, cols)), dtype='float')
To get the coassociation matrix from the labels, we simply call this function:
C = create_coassociation_matrix(labels)
From here, we can add multiple instances of these matrices together. This allows us
to combine the results from multiple runs of k-means. Printing out C (just enter C
into a new cell and run it) will tell you how many cells have nonzero values in them.
In my case, about half of the cells had values in them, as my clustering result had a
large cluster (the more even the clusters, the lower the number of nonzero values).
The next step involves the hierarchical clustering of the coassociation matrix. We will
do this by finding minimum spanning trees on this matrix and removing edges with
a weight lower than a given threshold.
[ 231 ]