Theory of Statistics - George Mason University

• monotone convergence:
for 0 ≤ X1 ≤ X2 · · · a.s.
• Fatou’s lemma:
Xn
1.5 Conditional Probability 113
a.s.
→ X ⇒ E(Xn|A) a.s.
→ E(X|A). (1.236)
0 ≤ Xn ∀ n ⇒ E(lim n inf Xn|A) ≤ lim n inf E(Xn|A) a.s.. (1.237)
• dominated convergence:
given a fixed Y with E(Y |A) < ∞,
|Xn| ≤ Y ∀ n and Xn a.s.
→ X ⇒ E(Xn|A) a.s.
→ E(X|A). (1.238)
Another useful fact is that if Y is A-measurable and |XY | and |X| are integrable
(notice this latter is stronger than what is required to define E(X|A)),
then
E(XY |A) = Y E(X|A) a.s. (1.239)
Some Useful Conditional Expectations
There are some conditional expectations that arise often, and which we should
immediately recognize. The simplest one is
E E(Y |X) = E(Y ). (1.240)
Note that the expectation operator is based on a probability distribution,
and so anytime we see “E”, we need to ask “with respect to what probability
distribution?” In notation such as that above, the distributions are implicit
and all relate to the same probability space. The inner expectation on the left
is with respect to the conditional distribution of Y given X, and so is a function
of X. The outer expectation is with respect to the marginal distribution
of X.
Approaching this slightly differently, we consider a random variable Z that
is a function of the random variables X and Y :
Z = f(X, Y ).
We have
E(f(X, Y )) = EY EX|Y (f(X, Y )|Y )
= EX EY |X(f(X, Y )|X) . (1.241)
Another useful conditional expectation relates adjusted variances to “total”
variances:
V(Y ) = V E(Y |X) + E V(Y |X) . (1.242)
This is intuitive, although you should be able to prove it formally.The intuitive
explanation is: the total variation in Y is the sum of the variation of its mean
given X and its average variation about X (or given X). (Think of SST =
SSR + SSE in regression analysis.)
This equality implies the Rao-Blackwell inequality (drop the second term
on the right).
Theory of Statistics c○2000–2013 James E. Gentle