Mathematical Methods for Physics and Engineering - Matematica.NET

31.5 MAXIMUM-LIKELIHOOD METHOD31.5.2 Transformation invariance and bias of ML estimatorsAn extremely useful property of ML estimators is that they are invariant toparameter transformations. Suppose that, instead of estimating some parametera of the assumed population, we wish to estimate some function α(a) oftheparameter. The ML estimator ˆα(a) is given by the value assumed by the functionα(a) at the maximum point of the likelihood, which is simply equal to α(â). Thus,we have the very convenient propertyˆα(a) =α(â).We do not have to worry about the distinction between estimating a and estimatinga function of a. Thisisnot true, in general, for other estimation procedures.◮A company measures the duration (in minutes) of the N intervals x i , i =1, 2,...,N,between successive telephone calls received by its switchboard. Suppose that the samplevalues x i are drawn independently from the distribution P (x|τ) =(1/τ)exp(−x/τ). FindtheML estimate of the parameter λ =1/τ.This is the same problem as the first one considered in subsection 31.5.1. In terms of thenew parameter λ, the log-likelihood function is given byN∑N∑ln L(x; λ) = ln(λe −λx i)= (ln λ − λx i ).i=1Differentiating with respect to λ and setting the result equal to zero, we have∂ ln LN∑( 1=i)∂λ λ − x =0.i=1Thus, the ML estimator of the parameter λ is given by( ) −11N∑ˆλ = x i = ¯x −1 . (31.75)Ni=1Referring back to (31.71), we see that, as expected, the ML estimators of λ and τ arerelated by ˆλ =1/ˆτ. ◭Although this invariance property is useful it also means that, in general, MLestimators may be biased. In particular, one must be aware of the fact that evenif â is an unbiased ML estimator of a it does not follow that the estimator ˆα(a) isalso unbiased. In the limit of large N, however, the bias of ML estimators alwaystends to zero. As an illustration, it is straightforward to show (see exercise 31.8)that the ML estimators ˆτ and ˆλ in the above example have expectation valuesE[ˆτ] =τ and E[ˆλ] = N λ. (31.76)N − 1In fact, since ˆτ = ¯x and the sample values are independent, the first result followsimmediately from (31.40). Thus, ˆτ is unbiased, but ˆλ =1/ˆτ is biased, albeit thatthe bias tends to zero for large N.1261i=1