General linear least squares - UGAstro

We can write a compact expression for the sum of the squares of the residuals,χ 2 = y − Ba 22, (5)where the notation ||…|| 2 is used to denote the Euclidian vector norm 1 ,x 2 2= x T 2x = ∑ x i, (6)and the superscript T denotes the transpose. Expanding Eq. (5) we findiχ 2 = ( y − Ba) T ( y − Ba)= y T y − 2y T Ba + a T B T Ba.(7)We want to minimize this expression as a function of a, so that the first derivatives with respectto a are zeroor∂χ 2∂a = −2BT y + 2B T Ba = 0 , (8)B T Ba = B T y . (9)Thus, the unknown vector a is found by multiplying each side by the inverse matrix (B T B) -1( B T B) −1 B T Ba = B T B( ) −1 B T y( ) −1 B T y.a = B T B(10)The quantity (B T B) -1 is known as the generalized or Moore-Penrose pseudo-inverse of B.Sophisticated versions of general least squares methods use singular value decomposition tocompute the inverse of B T B.1In two or three dimensions the Euclidian vector norm is just the magnitude (length) of thevector (think Pythagoras’ theorem).