Stat 5101 Lecture Notes - School of Statistics

5.1. RANDOM VECTORS 129Again like random vectors, the expectation or mean of a random matrix isa nonrandom matrix. If X is a random m × n matrix with elements X ij , thenthe mean of X is the matrix M with elementsµ ij = E(X ij ), (5.4)and we also write E(X) =Mto indicate all of the mn equations (5.4) with onematrix equation.5.1.4 Variance MatricesIn the preceding two sections we defined random vectors and random matricesand their expectations. The next topic is variances. One might think thatthe variance of a random vector should be similar to the mean, a vector havingcomponents that are the variances of the corresponding components of X, butit turns out that this notion is not useful. The reason is that variances andcovariances are inextricably entangled. We see this in the fact that the varianceof a sum involves both variances and covariances (Corollary 2.19 of these notesand the following comments). Thus the following definition.The variance matrix of an n-dimensional random vector X =(X 1 ,...,X n )is the nonrandom n × n matrix M having elementsm ij =cov(X i ,X j ). (5.5)As with variances of random scalars, we also use the notation var(X) for thevariance matrix. Note that the diagonal elements of M are variances becausethe covariance of a random scalar with itself is the variance, that is,m ii =cov(X i ,X i ) = var(X i ).This concept is well established, but the name is not. Lindgren calls Mthe covariance matrix of X, presumably because its elements are covariances.Other authors call it the variance-covariance matrix, because some of its elementsare variances too. Some authors, to avoid the confusion about variance,covariance, or variance-covariance, call it the dispersion matrix. In my humbleopinion, “variance matrix” is the right name because it is the generalization ofthe variance of a scalar random variable. But you’re entitled to call it what youlike. There is no standard terminology.Example 5.1.1.What are the mean vector and variance matrix of the random vector (X, X 2 ),where X is some random scalar? Letα k = E(X k )denote the ordinary moments of X. Then, of course, the mean and variance ofX are µ = α 1 andσ 2 = E(X 2 ) − E(X) 2 = α 2 − α 2 1,