Modeling and Multivariate Methods - SAS

Chapter 25 Comparing Model Performance 633
The Model Comparison Report
The Model Comparison Report
Figure 25.6 shows an example of the initial Model Comparison report for a continuous response.
Figure 25.6 Initial Model Comparison Report
The Predictors report shows all responses and all models being compared for each response. The fitting
platform that created the predictor column is also listed.
The Measures of Fit report shows measures of fit for each model. The columns are different for continuous
and categorical responses.
Measures of Fit for Continuous Responses
RSquare
The r-squared statistic.
RASE The root average squared error, the same value as RMSE except that RMSE adjusts for degrees of
freedom.
AAE
Freq
The average absolute error.
The column that contains frequency counts for each row.
Measures of Fit for Categorical Responses
Entropy RSquare One minus the ratio of the -log-likelihoods from the fitted model and the constant
probability model. It ranges from 0 to 1.
Generalized RSquare A generalization of the Rsquare measure that simplifies to the regular Rsquare for
continuous responses. Similar to the Entropy RSquare, but instead of using the log-likelihood, the
Generalized RSquare uses the 2/n root of the likelihood. The maximum value is 1 for a perfect model.
A value of 0 indicates that the model is no better than a constant model.
Mean -Log p
occurred.
The average of -log(p), where p is the fitted probability associated with the event that
RMSE The root mean square error, adjusted for degrees of freedom. For categorical responses, the
differences are between 1 and p (the fitted probability for the response level that actually occurred).