Data Mining Methods and Models

MULTICOLLINEARITY 121
TABLE 3.12 Regression Results, with Variance Inflation Factors Indicating a
Multicollinearity Problem
The regression equation is
Rating = 52.2 - 2.20 Sugars + 4.14 Fiber + 2.59 shelf 2
- 0.0421 Potass
Predictor Coef SE Coef T P VIF
Constant 52.184 1.632 31.97 0.000
Sugars -2.1953 0.1854 -11.84 0.000 1.4
Fiber 4.1449 0.7433 5.58 0.000 6.5
shelf 2 2.588 1.805 1.43 0.156 1.4
Potass -0.04208 0.02520 -1.67 0.099 6.7
S = 6.06446 R-Sq = 82.3% R-Sq(adj) = 81.4%
Analysis of Variance
Source DF SS MS F P
Regression 4 12348.8 3087.2 83.94 0.000
Residual Error 72 2648.0 36.8
Total 76 14996.8
Source DF Seq SS
Sugars 1 8701.7
Fiber 1 3416.1
shelf 2 1 128.5
Potass 1 102.5
Note that we need to standardize the variables involved in the composite, to
avoid the possibility that the greater variability of one of the variables will overwhelm
that of the other variable. For example, the standard deviation of fiber among all
cereals is 2.38 grams, and the standard deviation of potassium is 71.29 milligrams.
(The grams/milligrams scale difference is not at issue here. What is relevant is the
difference in variability, even on their respective scales.) Figure 3.11 illustrates the
difference in variability.
We therefore proceed to perform the regression of nutritional rating on the variables
sugarsz and shelf 2, and W = � �
fiberz + potassiumz /2. The results are provided
in Table 3.13.
Note first that the multicollinearity problem seems to have been resolved, with
the VIF values all near 1. Note also, however, that the regression results are rather
disappointing, with the values of R2 , R2 adj , and s all underperforming the model results
found in Table 3.7, from the model y = β0 + β1(sugars) + β2(fiber) + β4(shelf 2) +
ε, which did not even include the potassium variable.
What is going on here? The problem stems from the fact that the fiber variable
is a very good predictor of nutritional rating, especially when coupled with sugar
content, as we shall see later when we perform best subsets regression. Therefore,