Mathematical Methods for Physics and Engineering - Matematica.NET

STATISTICS31.5.3 Efficiency of ML estimatorsWe showed in subsection 31.3.2 that Fisher’s inequality puts a lower limit on thevariance V [â] of any estimator of the parameter a. Under our hypothesis H onp. 1255, the functional form of the population is given by the likelihood function,i.e. P (x|a,H)=L(x; a). Thus, if this hypothesis is correct, we may replace P byL in Fisher’s inequality (31.18), which then readsV [â] ≥(1+ ∂b∂a) 2 /E[ ]− ∂2 ln L∂a 2 ,where b is the bias in the estimator â. We usually denote the RHS by V min .An important property of ML estimators is that if there exists an efficientestimator â eff ,i.e.oneforwhichV [â eff ]=V min ,thenitmust be the ML estimatoror some function thereof. This is easily shown by replacing P by L in the proofof Fisher’s inequality given in subsection 31.3.2. In particular, we note that theequality in (31.22) holds only if h(x) =cg(x), where c is a constant. Thus, if anefficient estimator â eff exists, this is equivalent to demanding that∂ ln L= c[â eff − α(a)].∂aNow, the ML estimator â ML is given by∂ ln L∂a ∣ =0 ⇒ c[â eff − α(â ML )] = 0,a=âMLwhich, in turn, implies that â eff must be some function of â ML .◮Show that the ML estimator ˆτ given in (31.71) is an efficient estimator of the parameter τ.As shown in (31.70), the log-likelihood function in this case isN∑ (ln L(x; τ) =− ln τ + x )i.τi=1Differentiating twice with respect to τ, we find∂ 2 ln LN∑( 1=∂τ 2 τ − 2x ) ()i= N 1 − 2 N∑x 2 τ 3 τ 2 i , (31.77)τNi=1i=1and so the expectation value of this expression is[ ] ∂ 2 ln LE = N (1 − 2 )∂τ 2 τ 2 τ E[x i] = − N τ , 2where we have used the fact that E[x] =τ. Setting b = 0 in (31.18), we thus find that forany unbiased estimator of τ,V [ˆτ] ≥ τ2N .From (31.76), we see that the ML estimator ˆτ = ∑ i x i/N is unbiased. Moreover, usingthe fact that V [x] =τ 2 , it follows immediately from (31.40) that V [ˆτ] =τ 2 /N. Thus ˆτ is aminimum-variance estimator of τ. ◭1262