Building Machine Learning Systems with Python - Richert, Coelho

Dimensionality Reduction
However, the p-value basically tells us that whatever the correlation coefficient is,
we should not pay attention to it. The following output in the screenshot illustrates
the same:
In the first three cases that have high correlation coefficients, we would
probably want to throw out either or since they seem to convey similar
if not the same information.
In the last case, however, we should keep both features. In our application, this
decision would of course be driven by that p-value.
Although it worked nicely in the previous example, reality is seldom nice to us.
One big disadvantage of correlation-based feature selection is that it only detects
linear relationships (a relationship that can be modeled by a straight line). If we use
correlation on non-linear data, we see the problem. In the following example, we
have a quadratic relationship:
[ 224 ]