Theory of Statistics - George Mason University

488 6 Statistical Inference Based on Likelihood
pɛ(ɛ) = pɛ(y − Xβ), (6.59)
wrt a given σ-finite measure.
In the linear model (6.58), if ɛ ∼ N(0, σ 2 ), as we usually assume, we can
easily identify ηi, T(yi), and ζ(ηi) in equation (6.32), and of course, h(yi) ≡ 1.
This is a location-scale family.
Generalized Linear Models
A model as in equation (6.58) has limitations. Suppose, for example, that we
are interested in modeling a response that is binary, for example, two states of
a medical patient, “diseased” or “disease-free”. As usual, we set up a random
variable to map the sample space to IR:
Y : {disease-free,diseased} ↦→ {0, 1}.
The linear model X = Zβ + ɛ does not make sense. It is continuous and
unbounded.
A more useful model may address Pr(X = 0).
To make this more concrete, consider the situation in which several groups
of subjects are each administered a given dose of a drug, and the number
responding in each group is recorded. The data consist of the counts yi responding
in the ith group, which received a level xi of the drug.
A basic model is
P(Yi = 0|xi) = 1 − πi
(6.60)
P(Yi = 1|xi) = πi
The question is how does π depend on x?
A linear dependence, π = β0+β1x does not fit well in this kind of situation
– unless we impose restrictions, π would not be between 0 and 1.
We can try a transformation to [0, 1].
Suppose we impose an invertible function on
that will map it into [0, 1]:
or
η = β0 + β1x
We call this a link function.
A common model following this setup is
π = h(η), (6.61)
g(π) = η. (6.62)
πx = Φ(β0 + β1x), (6.63)
where Φ is the normal cumulative distribution function, and β0 and β1 are
unknown parameters to be estimated. This is called a probit model. The link
function in this case is Φ −1 .
Theory of Statistics c○2000–2013 James E. Gentle