Building Machine Learning Systems with Python - Richert, Coelho

Weights
Regression – Recommendations Improved
We can try a weighted average, multiplying each prediction by a given weight before
summing it all up. How do we find the best weights though? We learn them from the
data of course!
Ensemble learning
We are using a general technique in machine learning called ensemble
learning; this is not only applicable in regression. We learn an ensemble
(that is, a set) of predictors. Then, we combine them. What is interesting
is that we can see each prediction as being a new feature, and we are now
just combining features based on training data, which is what we have
been doing all along. Note that we are doing so for regression here, but
the same reasoning is applicable during classification: you learn how to
create several classifiers and a master classifier, which takes the output
of all of them and gives a final prediction. Different forms of ensemble
learning differ on how you combine the base predictors. In our case, we
reuse the training data that learned the predictors.
Data
Predict 1
Predict 2
Predict 3
....
Final
Prediction
By having a flexible way to combine multiple methods, we can simply try any idea
we wish by adding it into the mix of learners and letting the system give it a weight.
We can also use the weights to discover which ideas are good: if they get a high
weight, this means that it seems they are adding useful information. Ideas with
very low weights can even be dropped for better performance.
The code for this is very simple, and is as follows:
# We import the code we used in the previous examples:
import similar_movie
import corrneighbors
import usermodel
from sklearn.linear_model import LinearRegression
es = [
usermodel.estimate_all()
corrneighbors.estimate_all(),
similar_movie.estimate_all(),
]
[ 170 ]