Theory of Statistics - George Mason University

110 1 Probability Theory
A simple way of developing the ideas begins by defining the conditional
probability of event A, given event B. If Pr(B) = 0, the conditional probability
of event A given event B is
Pr(A|B) =
which leads to the useful multiplication rule
Pr(A ∩ B)
, (1.227)
Pr(B)
Pr(A ∩ B) = Pr(B)Pr(A|B). (1.228)
We see from this that if A and B are independent
Pr(A|B) = Pr(A).
If we interpret all of this in the context of the probability space (Ω, F, P),
we can define a new “conditioned” probability space, (Ω, F, PB), where we
define PB by
PB(A) = Pr(A ∩ B),
for any A ∈ F. From this conditional probability space we could then proceed
to develop “conditional” versions of the concepts discussed in the previous
sections.
This approach, however, is not entirely satisfactory because of the requirement
that Pr(B) = 0. More importantly, this approach in terms of events
does not provide a basis for the development of conditional probability density
functions.
Another approach is to make use of a concept of conditional expectation,
and that is what we will proceed to do. In this approach, we develop several
basic ideas before we finally speak of distributions of conditional random
variables in Section 1.5.4.
1.5.1 Conditional Expectation: Definition and Properties
The definition of conditional expectation of one random variable given another
random variable is developed in two stages. First, we define conditional
expectation over a sub-σ-field and consider some of its properties, and then
we define conditional expectation with respect to another measurable function
(a random variable, for example) in terms of the conditional expectation over
the sub-σ-field generated by the inverse image of the function.
A major difference in conditional expectations and unconditional expectations
is that conditional expectations may be nondegenerate random variables.
When the expectation is conditioned on a random variable, relations involving
the conditional expectations must be qualified as holding in probability,
or holding with probability 1.
Theory of Statistics c○2000–2013 James E. Gentle