Introduction to Categorical Data Analysis

138 BUILDING AND APPLYING LOGISTIC REGRESSION MODELS
5.1.1 How Many Predictors Can You Use?
Data are unbalanced on Y if y = 1 occurs relatively few times or if y = 0 occurs
relatively few times. This limits the number of predictors for which effects can be
estimated precisely. One guideline1 suggests there should ideally be at least 10 outcomes
of each type for every predictor. For example, if y = 1 only 30 times out of
n = 1000 observations, the model should have no more than about three predictors
even though the overall sample size is large.
This guideline is approximate. When not satisfied, software still fits the model. In
practice, often the number of variables is large, sometimes even of similar magnitude
as the number of observations. However, when the guideline is violated, ML estimates
may be quite biased and estimates of standard errors may be poor. From results to be
discussed in Section 5.3.1, as the number of model predictors increases, it becomes
more likely that some ML model parameter estimates are infinite.
Cautions that apply to building ordinary regression models hold for any GLM.
For example, models with several predictors often suffer from multicollinearity –
correlations among predictors making it seem that no one variable is important when
all the others are in the model. A variable may seem to have little effect because
it overlaps considerably with other predictors in the model, itself being predicted
well by the other predictors. Deleting such a redundant predictor can be helpful, for
instance to reduce standard errors of other estimated effects.
5.1.2 Example: Horseshoe Crabs Revisited
The horseshoe crab data set of Table 3.2 analyzed in Sections 3.3.2, 4.1.3, and 4.4.1
has four predictors: color (four categories), spine condition (three categories), weight,
and width of the carapace shell. We now fit logistic regression models using all these
to predict whether the female crab has satellites (males that could mate with her).
Again we let y = 1 if there is at least one satellite, and y = 0 otherwise.
Consider a model with all the main effects. Let {c1,c2,c3} be indicator variables
for the first three (of four) colors and let {s1,s2} be indicator variables for the first
two (of three) spine conditions. The model
logit[P(Y = 1)] =α + β1weight + β2width + β3c1 + β4c2 + β5c3 + β6s1 + β7s2
treats color and spine condition as nominal-scale factors. Table 5.1 shows the results.
A likelihood-ratio test that Y is jointly independent of these predictors simultaneously
tests H0: β1 =···=β7 = 0. The test statistic is −2(L0 − L1) = 40.6
with df = 7(P