PDF of Lecture Notes - School of Mathematical Sciences

1. DISTRIBUTION THEORY
Remark
This is also a re-statement of previously established results. To see this, observe that
if a T i is the i th row of A, we see that Y i = a T i X and, moreover, the (i, j) th element of
AΣA T = a T i Σa j = Cov ( a T i X, a T j X )
= Cov(Y i , Y j ).
1.9.4 Properties of variance matrices
If Σ = Var(X) for some random vector X = (X 1 , X 2 , . . . , X r ) T , then it must satisfy
certain properties.
Since Cov(X i , X j ) = Cov(X j , X i ), it follows that Σ is a square (r × r) symmetric
matrix.
Definition. 1.9.4
The square, symmetric matrix M is said to be positive definite [non-negative definite]
if a T Ma > 0 [≥ 0] for every vector a ∈ R r s.t. a ≠ 0.
=⇒ It is necessary and sufficient that Σ be non-negative definite in order that it can
be a variance matrix.
=⇒ To see the necessity of this condition, consider the linear combination a T X.
By Theorem 1.9.8, we have
=⇒ Σ must be non-negative definite.
0 ≤ Var(a T X) = a T Σa for every a;
Suppose λ is an eigenvalue of Σ, and let a be the corresponding eigenvector. Then
Σa = λa
=⇒ a T Σa = λa T a = λ||a|| 2 .
Hence Σ is non-negative definite (positive definite) iff its eigenvalues are all nonnegative
(positive).
If Σ is non-negative definite but not positive definite, then there must be at least one
zero eigenvalue. Let a be the corresponding eigenvector.
=⇒ a ≠ 0 but a T Σa = 0. That is, Var(a T X) = 0 for that a.
=⇒ the distribution of X is degenerate in the sense that either one of the X i ’s is
constant or else a linear combination of the other components.
56