Mathematical Methods for Physics and Engineering - Matematica.NET

31.6 THE METHOD OF LEAST SQUARESwhere {h 1 (x),h 2 (x),...,h M (x)} is some set of linearly independent fixed functionsof x, often called the basis functions. Note that the functions h i (x) themselves maybe highly non-linear functions of x. The ‘linear’ nature of the model (31.92) refersonly to its dependence on the parameters a i . Furthermore, in this case, it maybe shown that the LS estimators â i have zero bias and are minimum-variance,irrespective of the probability density function from which the measurement errorsn i are drawn.In order to obtain analytic expressions for the LS estimators â LS ,itisconvenientto write (31.92) in the formf(x; a) =M∑R ij a j , (31.93)where R ij = h j (x i ) is an element of the response matrix R of the experiment. Theexpression for χ 2 given in (31.90) can then be written, in matrix notation, asj=1χ 2 (a) =(y − Ra) T N −1 (y − Ra). (31.94)The LS estimates of the parameters a are now found, as shown in (31.91), bydifferentiating (31.94) with respect to the a i and setting the resulting expressionsequal to zero. Denoting by ∇χ 2 the vector with elements ∂χ 2 /∂a i , we find∇χ 2 = −2R T N −1 (y − Ra). (31.95)This can be verified by writing out the expression (31.94) in component form anddifferentiating directly.◮Verify result (31.95) by formulating the calculation in component form.To make the derivation less cumbersome, let us adopt the summation convention discussedin section 26.1, in which it is understood that any subscript that appears exactly twice inany term of an expression is to be summed over all the values that a subscript in thatposition can take. Thus, writing (31.94) in component form, we haveχ 2 (a) =(y i − R ik a k )(N −1 ) ij (y j − R jl a l ).Differentiating with respect to a p gives∂χ 2= −R ik δ kp (N −1 ) ij (y j − R jl a l )+(y i − R ik a k )(N −1 ) ij (−R jl δ lp )∂a p= −R ip (N −1 ) ij (y j − R jl a l ) − (y i − R ik a k )(N −1 ) ij R jp , (31.96)where δ ij is the Kronecker delta symbol discussed in section 26.1. By swapping the indicesi and j in the second term on the RHS of (31.96) and using the fact that the matrix N −1is symmetric, we obtain∂χ 2= −2R ip (N −1 ) ij (y j − R jk a k )∂a p= −2(R T ) pi (N −1 ) ij (y j − R jk a k ). (31.97)If we denote the vector with components ∂χ 2 /∂a p , p =1, 2,...,M,by∇χ 2 and write theRHS of (31.97) in matrix notation, we recover the result (31.95). ◭1273