Theory of Statistics - George Mason University

792 0 Statistical Mathematics
• Z T Z is nonnegative definite, and positive definite if and only if Z is of full
column rank;
• Z T Z = 0 =⇒ Z = 0.
The generalized inverses of Z T Z have useful properties. First, we see from
the definition, for any generalized inverse, (Z T Z) − that ((Z T Z) − ) T is also a
generalized inverse of Z T Z. (Note that (Z T Z) − is not necessarily symmetric.)
Another useful property of a Gramian matrix is that for any matrices B
and C (that are conformable for the operations indicated),
BZ T Z = CZ T Z ⇐⇒ BZ T = CZ T .
The implication from right to left is obvious, and we can see the left to right
implication by writing
(BZ T Z − CZ T Z)(B T − C T ) = (BZ T − CZ T )(BZ T − CZ T ) T ,
and then observing that if the left side is null, then so is the right side, and if
the right side is null, then BZ T − CZ T = 0. Similarly, we have
Also,
Z T ZB = Z T ZC ⇐⇒ Z T B = Z T C.
Z(Z T Z) − Z T Z = Z.
This means that (Z T Z) − Z T is a generalized inverse of Z
An important property of Z(Z T Z) − Z T is its invariance to the choice of
the generalized inverse of Z T Z. Suppose G is any generalized inverse of Z T Z.
Then we have
ZGZ T = Z(Z T Z) − Z T ;
that is, Z(Z T Z) − Z T is invariant to choice of generalized inverse.
The squared norm of the residual vector obtained from any generalized
inverse of Z T Z has some interesting properties. First, just by direct multiplication,
we get the “Pythagorean property” of the norm of the predicted
values and the residuals:
X − Zβ 2 = X − Z β 2 + Z β − Zβ 2
where β = (Z T Z) − Z T X for any generalized inverse. We also have
and
E(Z β) = Zβ,
E((Z β − Zβ) T (Z β − Zβ)) = V(Z β).
Because for any vector y, we have
y 2 = y T y = tr(y T y),
Theory of Statistics c○2000–2013 James E. Gentle