Learning Data Mining with Python

Classifying with scikit-learn Estimators
The Cosine distance is better suited to cases where some features are larger than
others and when there are lots of zeros in the dataset. Intuitively, we draw a line
from the origin to each of the samples, and measure the angle between those lines.
This can be seen in the following diagram:
In this example, each of the grey circles are in the same distance from the white
circle. In (a), the distances are Euclidean, and therefore, similar distances fit around
a circle. This distance can be measured using a ruler. In (b), the distances are
Manhattan, also called City Block. We compute the distance by moving across rows
and columns, similar to how a Rook (Castle) in Chess moves. Finally, in (c), we
have the Cosine distance that is measured by computing the angle between the lines
drawn from the sample to the vector, and ignore the actual length of the line.
The distance metric chosen can have a large impact on the final performance.
For example, if you have many features, the Euclidean distance between random
samples approaches the same value. This makes it hard to compare samples
as the distances are the same! Manhattan distance can be more stable in some
circumstances, but if some features have very large values, this can overrule lots of
similarities in other features. Finally, Cosine distance is a good metric for comparing
items with a large number of features, but it discards some information about the
length of the vector, which is useful in some circumstances.
For this chapter, we will stay with Euclidean distance, using other metrics in
later chapters.
[ 28 ]