Stat 5101 Lecture Notes - School of Statistics

132 Stat 5101 (Geyer) Course NotesThus independence is a sufficient but not necessary condition for (5.10) to hold.It is enough that the variables be uncorrelated.In statistics, our main interest is not in sums per se but rather in averagesX n = 1 n∑X i .(5.11a)ni=1The analogous formula for random vectors is just the same formula with boldfaceX n = 1 n∑X i .(5.11b)nWarning: the subscripts on the right hand side in (5.11b) do not indicatecomponents of a vector, rather X 1 , X 2 , ... is simply a sequence of randomvectors just as in (5.11a) X 1 , X 2 , ... is a sequence of random scalars. Theformulas for the mean and variance of a sum also give us the mean and varianceof an average.Theorem 5.1. If X 1 , X 2 , ... are random vectors having the same mean vectorµ, thenE(X n )=µ.(5.12a)If X 1 , X 2 , ... also have the same variance matrix M and are uncorrelated, theni=1var(X n )= 1 n M.(5.12b)This is exactly analogous to the scalar caseE(X n )=µ(5.13a)andvar(X n )= σ2(5.13b)nTheorem 5.2 (Alternate Variance and Covariance Formulas). If X andY are random vectors with means µ X and µ Y , thencov(X, Y) =E{(X−µ X )(Y − µ Y ) ′ }(5.14a)var(X) =E{(X−µ X )(Y − µ Y ) ′ }(5.14b)This hardly deserves the name “theorem” since it is obvious once one interpretsthe matrix notation. If X is m-dimensional and Y is n-dimensional, thenwhen we consider the vectors as matrices (“column vectors”) we see that thedimensions are(X − µ X ) (Y − µ Y ) ′m × 1 1 × nso the “sum” implicit in the matrix multiplication has only one term. Thus(5.14a) is the m × n matrix with i, j elementE{(X i − µ Xi )(Y j − µ Yj )} =cov(X i ,Y j )and hence is the covariance matrix cov(X, Y). Then we see that (5.14b) is justthe special case where Y = X.