Asymptotic Methods in Statistical Inference - Statistics Centre

19. Maximum likelihood: regularity, consistency
170
• Let X be the data, with density or mass function
(x). Viewed as a function of — (|x) =
(x ) — this is the ‘likelihood’ function.
The parameter value ˆ which maximizes (|x)
is viewed as that which makes the observed data
‘most likely to have occurred’. This value
ˆ =argmax(|x)
is the Maximum Likelihood Estimator (MLE). We
sometimes omit the dependence on x. Put () =
log (), the log-likelihood. Maximizing () is
generally easier than, and is of course equivalent
to, maximizing ().
• A more quantitative justification for the use of
the MLE is as follows.
Lemma: Assumethe are i.i.d. with density or
p.m.f. (), and that 0 isthetruevalue. Define
() ={X |( 0 |X) (|X)}. Then
0 ( ()) → 1as →∞ if 6= 0
(19.1)