Fundamentals of Probability and Statistics for Engineers

Â344 Fundamentals of Probability and Statistics for Engineersand (11.23) with the Cramer–Rao lower bounds defined in Section 9.2.2. Inorder to evaluate these lower bounds, a probability distribution of Y must bemade available. Without this knowledge, however, we can still show, in Theorem11.2, that the least squares technique leads to linear unbiased minimum-varianceestimators for and ; that is, among all unbiased estimators which are linearin Y , least-square estimators have minimum variance.Theorem 11.2: let random variable Y be defined by Equation (11.4). Givenasample (x 1 ,Y 1 ), (x 2 ,Y 2 ), . . . , (x n ,Y n ) of Y with its associated x values, leastgivenby Equation (11.17) are minimum variancesquare estimators ^A and ^Blinear unbiased estimatorsfor and , respectively.Proof of Theorem 11.2: the proof of this important theorem is sketchedbelow with use of vector–matrix notation.Consider a linear unbiased estimator of the formQ * ˆ‰…C T C† 1 C T ‡ GŠY:…11:24†We thus wish to prove that G ˆ 0 if Q * is to be minimum variance.The unbiasedness requirement leads to, in view of Equation (11.19),GC ˆ 0:…11:25†Consider now the covariance matrixUpon using Equations (11.19), (11.24), and (11.25) and expanding the covariance,we haveNow, in order to minimize the variances associated with the components of ,we must minimize each diagonal element of GG T . Since the iith diagonalelement of GG T is given bywhere g ij is the ijth element of G, we must haveand we obtaincovfQ * gˆEf…Q * q†…Q * q† T g: …11:26†covfQ * gˆ 2 ‰…C T C† 1 ‡ GG T Š:…GG T † ii ˆ Xnjˆ1g 2 ij ;g ij ˆ 0; for all i and j:Q *G ˆ 0:…11:27†TLFeBOOK