Theory of Statistics - George Mason University

426 5 Unbiased Point Estimation
The residual norm is minimized for t1 = R −1
1 c1 and t2 arbitrary. However, if
t2 = 0, then t2 is also minimized. Because U T b = t and U is orthogonal,
b2 = t2 = t12 + t22, and so with t2 = 0, that is, with b = β, β2 is
the minimum among the norms of all least squares solutions, b ∗ 2.
Quadratic Forms
Quadratic forms in nonnegative definite or positive definite matrices arise
often in statistical applications, especially in the analysis of linear models. The
analysis often involves the decomposition of a quadratic form in the positive
definite matrix A, y T Ay, into a sum, y T A1y + y T A2y, where A1 + A2 = A
and A1 and A2 are nonnegative definite matrices.
Cochran’s Theorems
There are various facts that are sometimes called Cochran’s theorem. The
simplest one concerns k symmetric idempotent n × n matrices, A1, . . ., Ak
that sum to the identity matrix.
Theorem 5.10 (Cochran’s theorem I)
Let A1, . . ., Ak be symmetric idempotent n × n matrices such that
Then
In = A1 + · · · + Ak.
AiAj = 0 for all i = j.
Proof.
For an arbitrary j, for some matrix V , we have
where r = rank(Aj). Now
which implies
V T AjV = diag(Ir, 0),
In = V T InV
k
= V T AiV
i=1
= diag(Ir, 0) +
V T AiV,
i=j
V T AiV = diag(0, In−r).
i=j
Now for each i, V T AiV is idempotent, and because the diagonal elements of
a symmetric idempotent matrix are all nonnegative, and hence the equation
Theory of Statistics c○2000–2013 James E. Gentle