Building Machine Learning Systems with Python - Richert, Coelho

Chapter 8
Now, we iterate over all the movies:
for i in range(nmovies):
movie_likeness[i] = all_correlations(reviews[:,i], reviews.T)
movie_likeness[i,i] = -1
We set the diagonal to -1; otherwise, the most similar movie to any movie is
itself, which is true, but very unhelpful. This is the same trick we used in Chapter
2, Learning How to Classify with Real-world Examples, when we first introduced the
nearest neighbor classification. Based on this matrix, we can easily write a function
that estimates a rating:
def nn_movie(movie_likeness, reviews, uid, mid):
likes = movie_likeness[mid].argsort()
# reverse the sorting so that most alike are in
# beginning
likes = likes[::-1]
# returns the rating for the most similar movie available
for ell in likes:
if reviews[u,ell] > 0:
return reviews[u,ell]
How well does the preceding function do? Fairly well: its RMSE is
only 0.85.
The preceding code does not show you all of the details of the crossvalidation.
While it would work well in production as it is written,
for testing, we need to make sure we have recomputed the likeness
matrix afresh without using the user that we are currently testing
on (otherwise, we contaminate the test set and we have an inflated
estimate of generalization). Unfortunately, this takes a long time, and
we do not need the full matrix for each user. You should compute only
what you need. This makes the code slightly more complex than the
preceding examples. On the companion website for this book, you will
find code with all the hairy details. There you will also find a much
faster implementation of the all_correlations function.
Combining multiple methods
We can now combine the methods given in the earlier section into a single
prediction. For example, we could average the predictions. This is normally good
enough, but there is no reason to think that both predictions are similarly good
and should thus have the exact same weight of 0.5. It might be that one is better.
[ 169 ]