Karjalainen, Pasi A. Regularization and Bayesian methods for ...

22 2. Estimation theory
Clearly, we have C xy = C T yx. The conditional density of x given y is defined to be
p(x|y) = p(x,y)
p(y)
(2.15)
whenever p(y) > 0 and 0 otherwise. Clearly, we can also write
p(y|x) = p(x,y)
p(x)
(2.16)
and we obtain
p(x|y)p(y) = p(y|x)p(x) (2.17)
This is called the Bayes’ theorem. The conditional mean of x given y is
η x|y = E {x|y} =
∫ ∞
−∞
xp(x|y)dx (2.18)
that is a function of the random variable y. If x and y are random vectors,
E {f(x,y)|y} =
∫ ∞
−∞
Using (2.10), (2.19) and (2.15) we can derive a useful result
E {f(x,y)} =
=
∫ ∞
−∞
∫ ∞
−∞
f(x,y)p(x|y)dx (2.19)
f(x,y)p(x,y)dxdy (2.20)
f(x,y)p(x|y)dxp(y)dy (2.21)
= E y {E {f(x,y)|y}} (2.22)
The conditional covariance of x given y is [152]
∣ }
C x|y = E
{(x − η x|y )(x − η x|y ) T ∣∣y
= E { xx ∣ T } y − ηx|y ηx|y T (2.23)
The random variables x i are said to be statistically independent if
p(x) = ∏ i
p i (x i ) (2.24)
and jointly Gaussian if their joint density can be written in the form
p(x) = (det C x) −1/2 (
exp − 1 )
(2π) n/2 2 (x − η x) T Cx −1 (x − η x )
(2.25)
where det C x denotes the determinant of C x . When x is jointly Gaussian with
mean η x and covariance C x , this is denoted by
x ∼ N(η x ,C x ) (2.26)