Building Machine Learning Systems with Python - Richert, Coelho

Learning How to Classify with Real-world Examples
Independent of what the original values were, after Z-scoring, a value of zero is the
mean and positive values are above the mean and negative values are below it.
Now every feature is in the same unit (technically, every feature is now
dimensionless; it has no units) and we can mix dimensions more confidently. In fact,
if we now run our nearest neighbor classifier, we obtain 94 percent accuracy!
Look at the decision space again in two dimensions; it looks as shown in the
following screenshot:
The boundaries are now much more complex and there is interaction between the
two dimensions. In the full dataset, everything is happening in a seven-dimensional
space that is very hard to visualize, but the same principle applies: where before a
few dimensions were dominant, now they are all given the same importance.
The nearest neighbor classifier is simple, but sometimes good enough. We can
generalize it to a k-nearest neighbor classifier by considering not just the closest
point but the k closest points. All k neighbors vote to select the label. k is typically
a small number, such as 5, but can be larger, particularly if the dataset is very large.
[ 46 ]