Building Machine Learning Systems with Python - Richert, Coelho

The only possibilities we have in this case is to either get more features, make the
model more complex, or change the model.
Chapter 5
Fixing high variance
If, on the contrary, we suffer from high variance that means our model is too
complex for the data. In this case, we can only try to get more data or decrease the
complexity. This would mean to increase k so that more neighbors would be taken
into account or to remove some of the features.
High bias or low bias
To find out what actually our problem is, we have to simply plot the train and test
errors over the data size.
High bias is typically revealed by the test error decreasing a bit at the beginning, but
then settling at a very high value with the train error approaching a growing dataset
size. High variance is recognized by a big gap between both curves.
Plotting the errors for different dataset sizes for 5NN shows a big gap between the
train and test error, hinting at a high variance problem. Refer to the following graph:
[ 103 ]