Theory of Statistics - George Mason University

116 1 Probability Theory
Proof. Exercise. Compare this with Theorem 1.13 on page 27, in which the
corresponding solution is g0(X) = E(Y |a) = E(Y ).
By the general definition of projection (Definition 0.0.9 on page 631), we
see that conditional expectation can be viewed as a projection in a linear space
defined by the square-integrable random variables over a given probability
space and the inner product 〈Y, X〉 = E(Y X) and its induced norm. (In fact,
some people define conditional expectation this way instead of the way we
have in Definitions 1.44 and 1.45.)
In regression applications in statistics using least squares, as we discuss on
page 434, “Y ”, or the “predicted” Y given X, that is, E(Y |X) is the projection
of Y onto X. For given fixed values of Y and X the predicted Y given X is
the vector projection, in the sense of Definition 0.0.9.
We now formally define projection for random variables in a manner analogous
to Definition 0.0.9. Note that the random variable space is the range of
the functions in G in Theorem 1.63.
Definition 1.46 (projection of a random variable onto a space of random variables)
Let Y be a random variable and let X be a random variable space defined on
the same probability space. A random variable Xp ∈ X such that
is called a projection of Y onto X.
E(Y − Xp2) ≤ E(Y − X2) ∀X ∈ X (1.245)
The most interesting random variable spaces are linear spaces, and in the
following we will assume that X is a linear space, and hence the norm arises
from an inner product so that the terms in inequality (1.245) involve variances
and covariances.
*** existence, closure of space in second norm (see page 35).
*** treat vector variables differently: E(Y − E(Y )2) is not the variance****
make this distinction earlier
When X is a linear space, we have the following result for projections.
Theorem 1.64
Let X be a linear space of random variables with finite second moments. Then
Xp is a projection of Y onto X iff Xp ∈ X and
Proof.
For any X, Xp ∈ X we have
E (Y − Xp) T X = 0 ∀X ∈ X. (1.246)
E((Y − X) T (Y − X)) = E((Y − Xp) T (Y − Xp))
+2E((Y − Xp) T (Xp − X))
+E((Xp − X) T (Xp − X))
Theory of Statistics c○2000–2013 James E. Gentle