an introduction to generalized linear models - GDM@FUDAN ...

3
Exponential Family and Generalized
Linear Models
3.1 Introduction
Linear models ofthe form
E(Yi) =µi = x T i β; Yi ∼ N(µi,σ 2 ) (3.1)
where the random variables Yi are independent are the basis ofmost
analyses ofcontinuous data. The transposed vector xT i represents the ith row
ofthe design matrix X. The example about the relationship between birthweight
and gestational age is ofthis form, see Section 2.2.2. So is the exercise
on plant growth where Yi is the dry weight ofplants and X has elements to
identify the treatment and control groups (Exercise 2.1). Generalizations of
these examples to the relationship between a continuous response and several
explanatory variables (multiple regression) and comparisons ofmore than two
means (analysis ofvariance) are also ofthis form.
Advances in statistical theory and computer software allow us to use methods
analogous to those developed for linear models in the following more
general situations:
1. Response variables have distributions other than the Normal distribution
– they may even be categorical rather than continuous.
2. Relationship between the response and explanatory variables need not be
ofthe simple linear form in (3.1).
One ofthese advances has been the recognition that many ofthe ‘nice’
properties ofthe Normal distribution are shared by a wider class ofdistributions
called the exponential family of distributions. These distributions
and their properties are discussed in the next section.
A second advance is the extension ofthe numerical methods to estimate the
parameters β from the linear model described in (3.1) to the situation where
there is some non-linear function relating E(Yi) =µi to the linear component
xT i β, that is
g(µi) =x T i β
(see Section 2.4). The function g is called the link function. In the initial formulation
ofgeneralized linear models by Nelder and Wedderburn (1972) and
in most ofthe examples considered in this book, g is a simple mathematical
function. These models have now been further generalized to situations where
functions may be estimated numerically; such models are called generalized
additive models (see Hastie and Tibshirani, 1990). In theory, the estimation
is straightforward. In practice, it may require a considerable amount of com-
© 2002 by Chapman & Hall/CRC