Fundamentals of Probability and Statistics for Engineers

290 Fundamentals of Probability and Statistics for EngineersAnalogous results are obtained when population X is discrete. Furthermore,the distribution of ^ tends to a normal distribution as n becomes large.This important result shows that MLE ^ is consistent. Since the variancegiven by Equation (9.104) is equal to the Cramér–Rao lower bound, it isefficient as n becomes large, or asymptotically efficient. The fact that MLE ^is normally distributed as n !1is also of considerable practical interest asprobability statements can be made regarding any observed value of a maximumlikelihood estimator as n becomes large.Let us remark, however, these important properties are large-sample properties.Unfortunately, very little can be said in the case of a small sample size; itmay be biased and nonefficient. This lack of reasonable small-sample propertiescan be explained in part by the fact that maximum likelihood estimation isbased on finding the mode of a distribution by attempting to select the trueparameter value. Estimators, in contrast, are generally designed to approachthe true value rather than to produce an exact hit. Modes are therefore not asdesirable as the mean or median when the sample size is small.Property 9.2: invariance property. It can be shown that, if ^ is the MLE of ,then the MLE of a function of , say g( ), is g( ^ ), where g( ) is assumed torepresent a one-to-one transformation and be differentiable with respect to .This important invariance property implies that, for example, if ^ is theMLE of the standard deviation in a distribution, then the MLE of thevariance 2 , c2 ,is^ 2 .Let us also make an observation on the solution procedure for solving likelihoodequations. Although it is fairly simple to establish Equation (9.99) orEquations (9.100), they are frequently highly nonlinear in the unknown estimates,and close-form solutions for the MLE are sometimes difficult, if not impossible,to achieve. In many cases, iterations or numerical schemes are necessary.Example 9.15. Let us consider Example 9.9 again and determine the MLEs ofmand 2 . The logarithm of the likelihood function isln L ˆ1 X n2 2jˆ1…x j m† 2 12 n ln 12 n ln 2: …9:105†2Let 1 ˆ m,and 2 ˆ 2 , as before; the likelihood equations areq ln Lq^ 1ˆ 1^ 2X njˆ1…x j^1 †ˆ0;q ln Lˆ 1 X n…xq^ 2 2^ 2 j^1 † 2 nˆ 0:2 jˆ12^ 2TLFeBOOK