Introduction to Categorical Data Analysis

5.1 STRATEGIES IN MODEL SELECTION 141
Table 5.2. Results of Fitting Several Logistic Regression Models to Horseshoe
Crab Data
Models Deviance
Model Predictors Deviance df AIC Compared Difference
1 C ∗ S + C ∗ W + S ∗ W 173.7 155 209.7 –
2 C + S + W 186.6 166 200.6 (2)–(1) 12.9 (df = 11)
3a C + S 208.8 167 220.8 (3a)–(2) 22.2 (df = 1)
3b S + W 194.4 169 202.4 (3b)–(2) 7.8 (df = 3)
3c C + W 187.5 168 197.5 (3c)–(2) 0.9 (df = 2)
4a C 212.1 169 220.1 (4a)–(3c) 24.6 (df = 1)
4b W 194.5 171 198.5 (4b)–(3c) 7.0 (df = 3)
5 C = dark + W 188.0 170 194.0 (5)–(3c) 0.5 (df = 2)
6 None 225.8 172 227.8 (6)–(5) 37.8 (df = 2)
Note: C = color, S = spine condition, W = width.
(category 4) and the others. The simpler model that has a single dummy variable
for color, equaling 0 for dark crabs and 1 otherwise, fits essentially as well [the
deviance difference between models (5) and (3c) equals 0.5, with df = 2]. Further
simplification results in large increases in the deviance and is unjustified.
5.1.5 AIC, Model Selection, and the “Correct” Model
In selecting a model, you should not think that you have found the “correct” one. Any
model is a simplification of reality. For example, you should not expect width to have
an exactly linear effect on the logit probability of satellites. However, a simple model
that fits adequately has the advantages of model parsimony. If a model has relatively
little bias, describing reality well, it provides good estimates of outcome probabilities
and of odds ratios that describe effects of the predictors.
Other criteria besides significance tests can help select a good model. The best
known is the Akaike information criterion (AIC). It judges a model by how close
its fitted values tend to be to the true expected values, as summarized by a certain
expected distance between the two. The optimal model is the one that tends to have its
fitted values closest to the true outcome probabilities. This is the model that minimizes
AIC =−2(log likelihood − number of parameters in model)
We illustrate this criterion using the models that Table 5.2 lists. For the model
C + W , having main effects of color and width, software (PROC LOGISTIC in
SAS) reports a −2 log likelihood value of 187.5. The model has five parameters –
an intercept and a width effect and three coefficients of dummy variables for color.
Thus, AIC = 187.5 + 2(5) = 197.5.
Of models in Table 5.2 using some or all of the three basic predictors, AIC is
smallest (AIC = 197.5) for C + W. The simpler model replacing C by an indicator