Theory of Statistics - George Mason University

6.4 Application: MLEs in Generalized Linear Models 485
Example 6.27 ML estimation of the variance in the one-way fixedeffects
AOV model
In Example 5.30, we assumed a normal distribution for the residuals, and
obtained the distribution of the sum of squares
SSE =
m
i=1 j=1
n
(Yij − Y i) 2 ,
and from that we obtained the UMVUE of σ 2 as SSE/m(n − 1).
From maximization of the likelihood, we obtain the MLE of σ 2 as
σ 2 = 1
nm
m
i=1 j=1
n
(Yij − Y i) 2
(6.49)
(exercise).
While the MLE of σ 2 is consistent in mean squared error as n → ∞
and m remains fixed, it is not consistent as m → ∞ and n remains fixed
(Exercise 6.3).
There are interesting ways of getting around the lack of consistency of the
variance estimator in Example 6.27. In the next example, we will illustrate
an approach that is a simple use of a more general method called REML, for
“residual maximum likelihood” (also called “restricted maximum likelihood”).
Example 6.28 REML estimation of the variance in the one-way
fixed-effects AOV model
In the preceding examples suppose there are only two observations per group;
that is, the model is
Yij = µ + αi + ɛij, i = 1, . . ., m; j = 1, 2,
with all of the other assumptions made above.
The MLE of σ 2 in equation (6.49) can be written as
σ 2 = 1
4m
m
2
i=1 j=1
(Yi1 − Yi2) 2 . (6.50)
We see that the limiting expectation of σ 2 as m → ∞ is σ/2; that is, the
estimator is not consistent. (This particular setup is called the “Neyman-Scott
problem”. In a fixed sample, of course, the estimator is biased, and there is
no reason to expect any change unless n instead of m were to increase.)
We see that the problem is caused by the unknown means, and as m
increases the number of unknown parameters increases linearly in m. We can,
however, reformulate the problem so as to focus on σ 2 . For i = 1, . . ., m,
Theory of Statistics c○2000–2013 James E. Gentle