Modeling and Multivariate Methods - SAS

Chapter 7 Performing Logistic Regression on Nominal and Ordinal Responses 203
The Logistic Fit Report
Prob>ChiSq is the probability of obtaining a greater Chi-square value by chance alone if the specified
model fits no better than the model that includes only intercepts.
RSquare (U) shows the R 2 , which is the ratio of the Difference to the Reduced negative log-likelihood
values. It is sometimes referred to as U, the uncertainty coefficient. RSquare ranges from zero for no
improvement to 1 for a perfect fit. A Nominal model rarely has a high Rsquare, and it has a Rsquare of
1 only when all the probabilities of the events that occur are 1.
AICc
BIC
is the corrected Akaike Information Criterion.
is the Bayesian Information Criterion
Observations
(or Sum Wgts) is the total number of observations in the sample.
Measure gives several measures of fit to assess model accuracy.
Entropy RSquare is the same as R-Square (U) explained above.
Generalized RSquare is a generalization of the Rsquare measure that simplifies to the regular Rsquare
for continuous normal responses. It is similar to the Entropy RSquare, but instead of using the
log-likelihood, it uses the 2/n root of the likelihood.
Mean -Log p is the average of -log(p), where p is the fitted probability associated with the event that
occurred.
RMSE is the root mean square error, where the differences are between the response and p (the fitted
probability for the event that actually occurred).
Mean Abs Dev is the average of the absolute values of the differences between the response and p (the
fitted probability for the event that actually occurred).
Misclassification Rate is the rate for which the response category with the highest fitted probability is
not the observed category.
For Entropy RSquare and Generalized RSquare, values closer to 1 indicate a better fit. For Mean -Log p,
RMSE, Mean Abs Dev, and Misclassification Rate, smaller values indicate a better fit.
After fitting the full model with two regressors in the ingots example, the –LogLikelihood on the Difference
line shows a reduction to 5.82 from the Reduced –LogLikelihood of 53.49. The ratio of Difference to
Reduced (the proportion of the uncertainty attributed to the fit) is 10.9% and is reported as the Rsquare
(U).
To test that the regressors as a whole are significant (the Whole Model test), a Chi-square statistic is
computed by taking twice the difference in negative log-likelihoods between the fitted model and the
reduced model that has only intercepts. In the ingots example, this Chi-square value is 2× 5.82 = 11.64 ,
and is significant at 0.003.
Lack of Fit Test (Goodness of Fit)
The next questions that JMP addresses are whether there is enough information using the variables in the
current model or whether more complex terms need to be added. The Lack of Fit test, sometimes called a
Goodness of Fit test, provides this information. It calculates a pure-error negative log-likelihood by
constructing categories for every combination of the regressor values in the data (Saturated line in the Lack
Of Fit table), and it tests whether this log-likelihood is significantly better than the Fitted model.