Springer - Read

376 Appendix A Random Variables and Probability Distributions
The conditional expectation of g(Y) given X x is then
E(g(Y)|X x)
∞
g(y)fY|X(y|x)dy.
−∞
If X and Y are independent, then fY|X(y|x) fY(y) by (A.2.2), and so the conditional
expectation of g(Y) given X x is
E(g(Y)|X x) E(g(Y)),
which, as expected, does not depend on x. The same ideas hold in the discrete case
with the probability mass function assuming the role of the density function.
Means and Covariances
If E|Xi| < ∞ for each i, then we define the mean or expected value of X
(X1,...,Xn) ′ to be the column vector
µX EX (EX1,...,EXn) ′ .
In the same way we define the expected value of any array whose elements are random
variables (e.g., a matrix of random variables) to be the same array with each random
variable replaced by its expected value (if the expectation exists).
If X (X1,...,Xn) ′ and Y (Y1,...,Ym) ′ are random vectors such that each
Xi and Yj has a finite variance, then the covariance matrix of X and Y is defined to
be the matrix
XY Cov(X, Y) E[(X − EX)(Y − EY) ′ ]
E(XY ′ ) − (EX)(EY) ′ .
The (i, j) element of XY is the covariance Cov(Xi,Yj) E(XiYj) − E(Xi)E(Yj).
In the special case where Y X,Cov(X, Y) reduces to the covariance matrix of the
random vector X.
Now suppose that Y and X are linearly related through the equation
Y a + BX,
where a is an m-dimensional column vector and B is an m × n matrix. Then Y has
mean
EY a + BEX (A.2.4)
and covariance matrix
YY BXXB ′
(see Problem A.3).
(A.2.5)
Proposition A.2.1 The covariance matrix XX of a random vector X is symmetric and nonnegative
definite, i.e., b ′ XXb ≥ 0 for all vectors b (b1,...,bn) ′ with real components.