Stat 5101 Lecture Notes - School of Statistics

232 Stat 5101 (Geyer) Course NotesThen‖x 1 + ···+x k ‖ 2 =(x 1 +···+x k ) ′ (x 1 +···+x k )k∑ k∑= x ′ ix j==i=1 j=1k∑x ′ ix ii=1k∑‖x i ‖ 2i=1because, by definition of orthogonality, all terms in the second line with i ≠ jare zero.We say an orthogonal set of vectors is orthonormal ifx ′ ix i =1.(E.2)That is, a set of vectors {x 1 ,...,x k }is orthonormal if it satisfies both (E.1) and(E.2).An orthonormal set is automatically linearly independent because ifthenk∑c i x i =0,i=1( k∑i=10=x ′ jc i x i)= c j x ′ jx j = c jholds for all j. Hence the only linear combination that is zero is the one withall coefficients zero, which is the definition of linear independence.Being linearly independent, an orthonormal set is always a basis for whateversubspace it spans. If we are working in n-dimensional space, and there are nvectors in the orthonormal set, then they make up a basis for the whole space.If there are k