Learning Data Mining with Python

Extracting Features with Transformers
The preceding function almost fits the interface needed to be used in scikit-learn's
univariate transformers. The function needs to accept two arrays (x and y in our
example) as parameters and returns two arrays, the scores for each feature and the
corresponding p-values. The chi2 function we used earlier only uses the required
interface, which allowed us to just pass it directly to SelectKBest.
The pearsonr function in SciPy accepts two arrays; however, the X array it accepts is
only one dimension. We will write a wrapper function that allows us to use this for
multivariate arrays like the one we have. Let's look at the code:
def multivariate_pearsonr(X, y):
We create our scores and pvalues arrays, and then iterate over each column of
the dataset:
scores, pvalues = [], []
for column in range(X.shape[1]):
We compute the Pearson correlation for this column only and the record both the
score and p-value.
cur_score, cur_p = pearsonr(X[:,column], y)
scores.append(abs(cur_score))
pvalues.append(cur_p)
The Pearson value could be between -1 and 1. A value of 1 implies a
perfect correlation between two variables, while a value of -1 implies a
perfect negative correlation, that is, high values in one variable give low
values in the other and vice versa. Such features are really useful to have,
but would be discarded. For this reason, we have stored the absolute
value in the scores array, rather than the original signed value.
Finally, we return the scores and p-values in a tuple:
return (np.array(scores), np.array(pvalues))
Now, we can use the transformer class as before to rank the features using the
Pearson correlation coefficient:
transformer = SelectKBest(score_func=multivariate_pearsonr, k=3)
Xt_pearson = transformer.fit_transform(X, y)
print(transformer.scores_)
[ 92 ]