Stat 5101 Lecture Notes - School of Statistics

136 Stat 5101 (Geyer) Course NotesCorollary 5.5. The variance matrix of any random vector is symmetric andpositive semi-definite.Proof. Symmetry follows from the symmetry property of covariances of randomscalars: cov(X i ,X j )=cov(X j ,X i ).The random scalar Y in Corollary 5.4 must have nonnegative variance. Thus(5.19b) implies c ′ var(X)c ≥ 0. Since c was an arbitrary vector, this provesvar(X) is positive semi-definite.The corollary says that a necessary condition for a matrix M to be thevariance matrix of some random vector X is that M be symmetric and positivesemi-definite. This raises the obvious question: is this a sufficient condition,that is, for any symmetric and positive semi-definite matrix M does there exista random vector X such that M = var(X)? We can’t address this question now,because we don’t have enough examples of random vectors for which we knowthe distributions. It will turn out that the answer to the sufficiency question is“yes.” When we come to the multivariate normal distribution (Section 5.2) wewill see that for any symmetric and positive semi-definite matrix M there is amultivariate normal random vector X such that M = var(X).A hyperplane in n-dimensional space R n is a set of the formH = { x ∈ R n : c ′ x = a } (5.22)for some nonzero vector c and some scalar a. We say a random vector X isconcentrated on the hyperplane H if P (X ∈ H) = 1. Another way of describingthe same phenomenon is to say that that H is a support of X.Corollary 5.6. The variance matrix of a random vector X is positive definiteif and only if X is not concentrated on any hyperplane.Proof. We will prove the equivalent statement that the variance matrix is notpositive definite if and only if is is concentrated on some hyperplane.First, suppose that M = var(X) is not positive definite. Then there is somenonzero vector c such that c ′ Mc = 0. Consider the random scalar Y = c ′ X.By Corollary 5.4 var(Y )=c ′ Mc = 0. Now by Corollary 2.34 of these notesY = µ Y with probability one. Since E(Y )=c ′ µ X by (5.19a), this says that Xis concentrated on the hyperplane (5.22) where a = c ′ µ X .Conversely, suppose that X is concentrated on the hyperplane (5.22). Thenthe random scalar Y = c ′ x is concentrated at the point a, and hence has variancezero, which is c ′ Mc by Corollary 5.4. Thus M is not positive definite.5.1.9 Degenerate Random VectorsRandom vectors are sometimes called degenerate by those who believe inthe kindergarten principle of calling things we don’t like bad names. And whywouldn’t we like a random vector concentrated on a hyperplane? Because itdoesn’t have a density. A hyperplane is a set of measure zero, hence any integralover the hyperplane is zero and cannot be used to define probabilities andexpectations.