Building Machine Learning Systems with Python - Richert, Coelho

As we have learned previously, we will first have to vectorize this post before we
predict its label as follows:
>>> new_post_vec = vectorizer.transform([new_post])
>>> new_post_label = km.predict(new_post_vec)[0]
Chapter 3
Now that we have the clustering, we do not need to compare new_post_vec to all
post vectors. Instead, we can focus only on the posts of the same cluster. Let us fetch
their indices in the original dataset:
>>> similar_indices = (km.labels_==new_post_label).nonzero()[0]
The comparison in the bracket results in a Boolean array, and nonzero converts that
array into a smaller array containing the indices of the True elements.
Using similar_indices, we then simply have to build a list of posts together with
their similarity scores as follows:
>>> similar = []
>>> for i in similar_indices:
... dist = sp.linalg.norm((new_post_vec - vectorized[i]).toarray())
... similar.append((dist, dataset.data[i]))
>>> similar = sorted(similar)
>>> print(len(similar))
44
We found 44 posts in the cluster of our post. To give the user a quick idea of what
kind of similar posts are available, we can now present the most similar post (show_
at_1), the least similar one (show_at_3), and an in-between post (show_at_2), all of
which are from the same cluster as follows:
>>> show_at_1 = similar[0]
>>> show_at_2 = similar[len(similar)/2]
>>> show_at_3 = similar[-1]
[ 69 ]