Applied Statistics Using SPSS, STATISTICA, MATLAB and R

304 7 Data Regression
the same way as we performed feature selection in Chapter 6. The search
procedure has also to use an appropriate criterion for predictor selection. There are
many such criteria published in the literature. We indicate here just a few:
– SSE (minimisation)
– R square (maximisation)
– t statistic (maximisation)
– F statistic (maximisation)
When building the model, these criteria can be used in a stepwise manner the
same way as we performed sequential feature selection in Chapter 6. That is, by
either adding consecutive variables to the model − the so-called forward search
method −, or by removing variables from an initial set − the so-called backward
search method.
For instance, a very popular method is to use forward stepwise building up the
model using the F statistic, as follows:
1. Initially enters the variable, say X1, that has maximum Fk
MSR(Xk)/MSE(Xk), which must be above a certain specified level.
=
2. Next is added the variable with maximum Fk = MSR(Xk | X1) / MSE(Xk, X1)
and above a certain specified level.
3. The Step 2 procedure goes on until no variable has a partial F above the
specified level.
Example 7.17
Q: Apply the forward stepwise procedure to the foetal weight data (see Example
7.13), using as initial predictor sets {BPD, CP, AP} and {MW, MH, BPD, CP, AP,
FL}.
A: Figure 7.11 shows the evolution of the model using the forward stepwise
method to {BPD, CP, AP}. The first variable to be included, with higher F, is the
variable AP. The next variables that are included have a decreasing F contribution
but still higher than the specified level of “F to Enter”, equal to 1. These results
confirm the findings on partial correlation coefficients discussed in section 7.2.5
(Table 7.4).
Figure 7.11. Forward stepwise regression (obtained with STATISTICA) for the
foetal weight example, using {BPD, CP, AP} as initial predictor set.