Introduction to Categorical Data Analysis

3.5 FITTING GENERALIZED LINEAR MODELS 89
3.5.2 Wald, Likelihood-Ratio, and Score Inference Use
the Likelihood Function
Figure 3.7 shows a generic plot of a log-likelihood function L for a parameter β.
This plot illustrates the Wald, likelihood-ratio, and score tests of H0: β = 0. The
log-likelihood function for some GLMs, including binomial logistic regression
models and Poisson loglinear models, has concave (bowl) shape. The ML estimate ˆβ is
the point at which the log-likelihood takes its highest value. The Wald test is based on
the behavior of the log-likelihood function at the ML estimate ˆβ, having chi-squared
form ( ˆβ/SE) 2 . The SE of ˆβ depends on the curvature of the log-likelihood function
at the point where it is maximized, with greater curvature giving smaller SE values.
The score test is based on the behavior of the log-likelihood function at the null
value for β of 0. It uses the size of the derivative (slope) of the log-likelihood function,
evaluated at the null hypothesis value of the parameter. The derivative at β = 0 tends
to be larger in absolute value when ˆβ is further from that null value. The score statistic
also has an approximate chi-squared distribution with df = 1. We shall not present
the general formula for score statistics, but many test statistics in this text are this
type. An example is the Pearson statistic for testing independence. An advantage of
the score statistic is that it exists even when the ML estimate ˆβ is infinite. In that case,
one cannot compute the Wald statistic.
The likelihood-ratio test combines information about the log-likelihood function
both at ˆβ and at the null value for β of 0. It compares the log-likelihood values L1
at ˆβ and L0 at β = 0 using the chi-squared statistic −2(L0 − L1). In Figure 3.7, this
statistic is twice the vertical distance between values of the log-likelihood function at
ˆβ and at β = 0. In a sense, this statistic uses the most information of the three types
of test statistic. It is usually more reliable than the Wald statistic, especially when n
is small to moderate.
Figure 3.7. Information used in Wald, likelihood-ratio, and efficient score tests.