Stat 5101 Lecture Notes - School of Statistics

3.5. CONDITIONAL EXPECTATION AND PREDICTION 107The other important consequence of (3.36) is obtained by taking a(X) =E(Y)=µ Y (that is, a is the constant function equal to µ Y ). This givesE{[Y − µ Y ] 2 } = E{var(Y | X)} + E{[µ Y − E(Y | X)] 2 } (3.37)The left hand side of (3.37) is, by definition var(Y ). By the iterated expectationaxiom, E{E(Y | X)} = E(Y )=µ Y , so the second term on the right hand sideis the expected squared deviation of E(Y | X) from its expectation, which is,by definition, its variance. Thus we have obtained the following theorem.Theorem 3.7 (Iterated Variance Formula). If Y ∈ L 2 ,var(Y )=E{var(Y | X)} +var{E(Y |X)}.Example 3.5.2 (Example 3.3.1 Continued).Suppose X 0 , X 1 , ... is an infinite sequence of identically distributed randomvariables, having mean E(X i )=µ X and variance var(X i )=σX 2 , and suppose Nis a nonnegative integer-valued random variable independent of the X i havingmean E(N) =µ N and variance var(N) =σN 2 . Note that we have now tiedup the loose end in Example 3.3.1. We now know from Theorem 3.4 thatindependence of the X i and N impliesE(X i | N) =E(X i )=µ X .and similarlyvar(X i | N )=var(X i )=σX.2Question: What is the variance ofS N = X 1 + ···+X Nexpressed in terms of the means and variances of the X i and N?This is easy using the iterated variance formula. First, as we found in Example3.3.1,E(S N | N) =NE(X i | N)=Nµ X .A similar calculation givesvar(S N | N) =Nvar(X i | N )=Nσ 2 X(because of the assumed independence of the X i and N). Hencevar(S N )=E{var(S N | N )} +var{E(S N |N)}=E(NσX) 2 + var(Nµ X )= σXE(N)+µ 2 2 Xvar(N)= σXµ 2 N + µ 2 XσN2Again notice that it is impossible to do this problem any other way. Thereis not enough information given to use any other approach.Also notice that the answer is not exactly obvious. You might just guess,using your intuition, the answer to Example 3.3.1. But you wouldn’t guess this.You need the theory.