Stat 5101 Lecture Notes - School of Statistics

106 Stat 5101 (Geyer) Course NotesTheorem 3.5 (Conditional Parallel Axis Theorem). If Y ∈ L 1E{[Y − a(X)] 2 | X} = var(Y | X)+[a(X)−E(Y |X)] 2 (3.35)The argument is exactly the same as that given for the unconditional version,except for the need to use Axiom CE1 instead of Axiom E2 to pull a functionof the conditioning variable out of the conditional expectation. Otherwise, onlythe notation changes.If we take the unconditional expectation of both sides of (3.35), we getE ( E{[Y − a(X)] 2 | X} ) = E{var(Y | X)} + E{[a(X) − E(Y | X)] 2 }and by the iterated expectation axiom, the left hand side is the the unconditionalexpectation, that is,E{[Y − a(X)] 2 } = E{var(Y | X)} + E{[a(X) − E(Y | X)] 2 } (3.36)This relation has no special name, but it has two very important special cases.The first is the prediction theorem.Theorem 3.6. For predicting a random variable Y given the value of anotherrandom variable X, the predictor function a(X) that minimizes the expectedsquared prediction errorE{[Y − a(X)] 2 }is the conditional expectation a(X) =E(Y |X).The proof is extremely simple. The expected squared prediction error isthe left hand side of (3.36). On the right hand side of (3.36), the first termdoes not contain a(X). The second term is the expectation of the square ofa(X) − E(Y | X). Since a square is nonnegative and the expectation of anonnegative random variable is nonnegative (Axiom E1), the second term isalways nonnegative and hence is minimized when it is zero. By Theorem 2.32,that happens if and only if a(X) = E(Y | X) with probability one. (Yetanother place where redefinition on a set of probability zero changes nothing ofimportance).Example 3.5.1 (Best Prediction).Suppose X and Y have the unnormalized joint densityh(x, y) =(x+y)e −x−y ,x > 0,y>0,what function of Y is the best predictor of X in the sense of minimizing expectedsquared prediction error?The predictor that minimizes expected squared prediction error is the regressionfunctiona(Y )=E(X|Y)= 2+Y1+Yfound in Example 3.4.5.