Karjalainen, Pasi A. Regularization and Bayesian methods for ...

24 2. Estimation theory
2.5 Bayes cost method
If we assume that θ is a random vector with a known joint density p(θ,z) with
the observation z, we have made the so-called Bayesian assumption [142]. This
assumption leads to the so-called Bayes cost method for solving the estimator
ˆθ(z). For the cost method we define the function C(θ, ˆθ) that assigns to each
combination of actual parameter value and estimate the unique real valued cost.
We call C(θ, ˆθ) the cost function. The expected value of the cost is given by
{
B(ˆθ) = E C(θ, ˆθ(z))
}
=
∫ ∞ ∫ ∞
−∞
−∞
C(θ, ˆθ(z))p(θ,z)dθ dz (2.30)
that is called the Bayes cost. From (2.15) we obtain p(θ,z) = p(z|θ)p(θ) and the
expectation can be written in the form
∫ ∞
(∫ ∞
)
B(ˆθ) = C(θ, ˆθ(z))p(z|θ)dz p(θ)dθ (2.31)
−∞ −∞
The inner integral is clearly the conditional expectation of the cost given θ and we
write
∫ ∞
B(ˆθ|θ) = C(θ, ˆθ(z))p(z|θ)dz (2.32)
−∞
{
= E C(θ, ˆθ)
}
∣
∣θ
(2.33)
This can be called the conditional Bayes cost, given θ, and in terms of the conditional
cost the Bayes cost of the estimator can be written as
∫ ∞
B(ˆθ) = B(ˆθ|θ)p(θ)dθ (2.34)
−∞
{
= E θ
{E C(θ, ˆθ)
}}
∣
∣θ
(2.35)
}
= E θ
{B(ˆθ|θ)
(2.36)
Similarily using p(θ,z) = p(θ|z)p(z) we can write
{
B(ˆθ) = E C(θ, ˆθ(z))
}
{
= E z
{E C(θ, ˆθ)
}}
∣
∣z
}
= E z
{B(ˆθ|z)
(2.37)
(2.38)
(2.39)
where
∫ ∞
B(ˆθ|z) = C(θ, ˆθ(z))p(θ|z)dθ (2.40)
−∞
{
= E C(θ, ˆθ)
}
∣
∣z
(2.41)