Stat 5101 Lecture Notes - School of Statistics

5.2. THE MULTIVARIATE NORMAL DISTRIBUTION 141If M = var(X) has elements m ij , thenc ij =m ij√mii m jjNote that the diagonal elements c ii of a correlation matrix are all equal to one,because the correlation of any random variable with itself is one.Theorem 5.8. Every correlation matrix is positive semi-definite. The correlationmatrix of a random vector X is positive definite if and only the variancematrix of X is positive definite.Proof. This follows from the analogous facts about variance matrices.It is important to understand that the requirement that a variance matrix(or correlation matrix) be positive semi-definite is a much stronger requirementthan the correlation inequality (correlations must be between −1 and +1). Thetwo requirements are related: positive semi-definiteness implies the correlationinequality, but not vice versa. That positive semi-definiteness implies the correlationinequality is left as an exercise (Problem 5-4). That the two conditionsare not equivalent is shown by the following example.Example 5.1.6 (All Correlations the Same).Suppose X =(X 1 ,...,X n ) is a random vector and all the components havethe same correlation, as would be the case if the components are exchangeablerandom variables, that is, cor(X i ,X j )=ρfor all i and j with i ≠ j. Thenthe correlation matrix of X has one for all diagonal elements and ρ for all offdiagonalelements. In Problem 2-22 it is shown that positive definiteness of thecorrelation matrix requires− 1n − 1 ≤ ρ.This is an additional inequality not implied by the correlation inequality.The example says there is a limit to how negatively correlated a sequenceof exchangeable random variables can be. But even more important than thisspecific discovery, is the general message that there is more to know aboutcorrelations than that they are always between −1 and +1. The requirementthat a correlation matrix (or a variance matrix) be positive semi-definite is muchstronger. It implies a lot of other inequalities. In fact it implies an infinite familyof inequalities: M is positive semi-definite only if c ′ Mc ≥ 0 for every vectorc. That’s a different inequality for every vector c and there are infinitely manysuch vectors.5.2 The Multivariate Normal DistributionThe standard multivariate normal distribution is the distribution of the randomvector Z =(Z 1 ,...,Z n ) having independent and identically N (0, 1) dis-