Applied Statistics Using SPSS, STATISTICA, MATLAB and R

320 7 Data Regression
Besides its use in the selection of “smooth”, non-over-fitted models, ridge
regression is also used as a remedy to decrease the effects of multicollinearity as
illustrated in the following Example 7.20. In this application one must select a
ridge factor corresponding to small values of the VIF factors.
a
2.5
2
1.5
1
0.5
g(x)
x
0
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
2.5
2
1.5
1
0.5
0
g(x)
x
b
2.5
2
1.5
1
0.5
g(x)
x
0
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
c 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 d 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Figure 7.20. Fitting a second-order model to a very simple dataset (3 points
represented by solid circles) with ridge factor: a) 0; b) 0.6; c) 1; d) 50.
2
1.8
1.6
1.4
1.2
1
0.8
0.6
0.4
0.2
0
E
Figure 7.21. SSE (solid line) and SSE(L) (dotted line) curves for the ridge
regression solutions of Figure 7.20 dataset.
2.5
2
1.5
1
0.5
0
g(x)
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
r
x