Building Machine Learning Systems with Python - Richert, Coelho

Dimensionality Reduction
Of course, using MDS requires an understanding of the individual feature's units;
maybe we are using features that cannot be compared using the Euclidean metric.
For instance, a categorical variable, even when encoded as an integer (0 = red circle,
1 = blue star, 2 = green triangle, and so on), cannot be compared using Euclidean (is
red closer to blue than to green?).
But once we are aware of this issue, MDS is a useful tool that reveals similarities in
our data that otherwise would be difficult to see in the original feature space.
Looking a bit deeper into MDS, we realize that it is not a single algorithm, but
a family of different algorithms, of which we have used just one. The same was
true for PCA, and in case you realize that neither PCA nor MDS solves your
problem, just look at the other manifold learning algorithms that are available
in the Scikit-learn toolkit.
Summary
We learned that sometimes we can get rid of all the features using feature selection
methods. We also saw that in some cases this is not enough, and we have to employ
feature extraction methods that reveal the real and the lower-dimensional structure
in our data, hoping that the model has an easier game with it.
We have only scratched the surface of the huge body of available dimensionality
reduction methods. Still, we hope that we have got you interested in this whole
field, as there are lots of other methods waiting for you to pick up. At the end,
feature selection and extraction is an art, just like choosing the right learning
method or training model.
The next chapter covers the use of Jug, a little Python framework to manage
computations in a way that takes advantage of multiple cores or multiple
machines. We will also learn about AWS – the Amazon cloud.
[ 240 ]