Springer - Read

290 Chapter 8 State-Space Models
The last equality follows from the fact that
0 ∂
∂
∂
(1) (f (x|Y; θ)dx f(x|Y; θ) dx.
∂θ ∂θ
θ ˆθ ∂θ θ ˆθ
The computational advantage of the EM algorithm over direct maximization of the
likelihood is most pronounced when the calculation and maximization of the exact
likelihood is difficult as compared with the maximization of Q in the M-step. (There
are some applications in which the maximization of Q can easily be carried out
explicitly.)
Missing Data
The EM algorithm is particularly useful for estimation problems in which there are
missing observations. Suppose the complete data set consists of Y1,...,Yn of which
r are observed and n − r are missing. Denote the observed and missing data by Y
(Yi1,...,Yir ) ′ and X (Yj1,...,Yjn−r ) ′ , respectively. Assuming that W (X ′ , Y ′ ) ′
has a multivariate normal distribution with mean 0 and covariance matrix , which
depends on the parameter θ, the log-likelihood of the complete data is given by
ℓ(θ; W) − n 1
1
ln(2π)− ln det() −
2 2 2 W′ W.
The E-step requires that we compute the expectation of ℓ(θ; W) with respect to the
conditional distribution of W given Y with θ θ (i) . Writing (θ) as the block matrix
11 12
,
21 22
which is conformable with X and
Y, the conditional distribution
of
W given Y is
11|2(θ) 0
and covariance matrix , where ˆX
multivariate normal with mean ˆX
0
0 0
Eθ(X|Y) 12 −1
22 Y and 11|2(θ) 11 − 12 −1
22 21 (see Proposition A.3.1).
Using Problem A.8, we have
Eθ (i)
where ˆW
(X ′ , Y ′ ) −1 (θ)(X ′ , Y ′ ) ′ |Y trace 11|2(θ (i) ) −1
11|2 (θ) + ˆW ′ −1 (θ) ˆW,
ˆX ′ , Y ′
′
. It follows that
Q θ|θ (i)
ℓ θ, ˆW − 1
2 trace 11|2
θ (i) −1
11|2 (θ) .
The first term on the right is the log-likelihood based on the complete data, but with
X replaced by its “best estimate” ˆX calculated from the previous iteration. If the
increments θ (i+1) − θ (i) are small, then the second term on the right is nearly constant
(≈ n − r) and can be ignored. For ease of computation in this application we shall
use the modified version
˜Q θ|θ (i)
ℓ θ; ˆW .