Modeling and Multivariate Methods - SAS

Chapter 4 Fitting Stepwise Regression Models 141
The Stepwise Report
Table 4.4 Description of Current Model Statistics (Continued)
RSquare
RSquare Adj
Cp
Proportion of the variation in the response that can be attributed to terms in
the model rather than to random error.
Adjusts R 2 to make it more comparable over models with different numbers
of parameters by using the degrees of freedom in its computation. The
adjusted R 2 is useful in stepwise procedure because you are looking at many
different models and want to adjust for the number of terms in the model.
Mallow’s C p criterion for selecting a model. It is an alternative measure of
total squared error and can be defined as follows:
C p
=
SSE p
-----------
s 2 – ( N – 2p)
where s 2 is the MSE for the full model and SSE p is the sum-of-squares error
for a model with p variables, including the intercept. Note that p is the
number of x-variables+1. If C p is graphed with p, Mallows (1973)
recommends choosing the model where C p first approaches p.
p
AICc
Number of parameters in the model, including the intercept.
Corrected Akaike’s Information Criterion, defined as follows:
AICc = -2loglikelihood + 2k( k+
1)
2k + ---------------------- n – k – 1
where k is the number of estimated parameters, including intercept and error
terms in the model, and n is the number of observations in the data set.
Burnham and Anderson (2004) discuss using AIC c for model selection. The
best model has the smallest value, as discussed in Akaike (1974).
BIC
Bayesian Information Criterion defined as
-2loglikelihood + k ln(n)
where k is the number of parameters, and n is the sample size.
Forward Selection Example
In forward selection, terms are entered into the model and most significant terms are added until all of the
terms are significant.
1. Complete the steps in “Example Using Stepwise Regression” on page 135.
Notice that the default selection for Direction is Forward.
2. Click Step.
From the top figure in Figure 4.4, you can see that after one step, the most significant term, Runtime, is
entered into the model.