Fundamentals of Probability and Statistics for Engineers

9Parameter EstimationSuppose that a probabilistic model, represented by probability density function(pdf) f(x), has been chosen for a physical or natural phenomenon for whichparameters 1, 2, . . . are to be estimated from independently observed datax 1 ,x 2 ,...,x n . Let us consider for a moment a single parameter for simplicityand writef(x; ) to mean a specified probability distribution where is the unknownparameter to be estimated. The parameter estimation problem is then one ofdetermining an appropriate function of x 1 ,x 2 ,...,x n ,say h(x 1 ,x 2 ,...,x n ), whichgives the ‘best’ estimate of . In order to develop systematic estimation procedures,we need to make more precise the terms that were defined rather loosely in thepreceding chapter and introduce some new concepts needed for this development.9.1 SAMPLES AND STATISTICSGiven an independent data set x 1 ,x 2 ,...,x n ,let^ ˆ h…x 1 ; x 2 ; ...; x n †…9:1†be an estimate of parameter . In order to ascertain its general properties, it isrecognized that, if the experiment that yielded the data set were to be repeated,we would obtain different values for x 1 ,x 2 ,...,x n . The function h(x 1 ,x 2 ,...,x n )when applied to the new data set would yield a different value for ^ .Wethusseethat estimate ^ is itself a random variable possessing a probability distribution,which depends both on the functional form defined by h and on the distributionof the underlying random variable X. The appropriate representation of ^ is thus^ ˆ h…X 1 ; X 2 ; ...; X n †;…9:2†where X 1 ,X 2 ,...,X n are random variables, representing a sample from randomvariable X, which is referred to in this context as the population. In practicallyFundamentals of Probability and Statistics for Engineers T.T. Soong© 2004 John Wiley & Sons, LtdISBNs: 0-470-86813-9 (HB) 0-470-86814-7 (PB)TLFeBOOK