Building Machine Learning Systems with Python - Richert, Coelho

Regression –
Recommendations
You have probably learned about regression already in high school mathematics
class, this was probably called ordinary least squares (OLS) regression then. This
centuries old technique is fast to run and can be effectively used for many real-world
problems. In this chapter, we will start by reviewing OLS regression and showing
you how it is available in both NumPy and scikit-learn.
In various modern problems, we run into limitations of the classical methods
and start to benefit from more advanced methods, which we will see later in this
chapter. This is particularly true when we have many features, including when we
have more features than examples (which is something that ordinary least squares
cannot handle correctly). These techniques are much more modern, with major
developments happening in the last decade. They go by names such as lasso, ridge,
or elastic nets. We will go into these in detail.
Finally, we will start looking at recommendations. This is an important area in many
applications as it is a significant added-value to many applications. This is a topic
that we will start exploring here and will see in more detail in the next chapter.
Predicting house prices with regression
Let us start with a simple problem, predicting house prices in Boston.
We can use a publicly available dataset. We are given several demographic and
geographical attributes, such as the crime rate or the pupil-teacher ratio, and the
goal is to predict the median value of a house in a particular area. As usual, we
have some training data, where the answer is known to us.