Introduction to the EM algorithm - Department of Statistics

More documents

Recommendations

Info

Taking the expectation on both sides of the previous equation we have that ∫ log p(y|θ)p(z|y, θ (k) )dz = ∫ ∫ log p(y, z|θ)p(z|y, θ (k) )dz − log p(z|y, θ)p(z|y, θ (k) )dz. Therefore, log p(y|θ) = Q(θ, θ (k) ) − H(θ, θ (k) ). Camila Souza (UBC) Introduction to the EM algorithm November 2010 10 / 25
How can we find a value θ (k+1) so that log p(y|θ (k+1) ) ≥ log p(y|θ (k) )? Using log p(y|θ) = Q(θ, θ (k) ) − H(θ, θ (k) ) we have Q(θ (k+1) , θ (k) ) − Q(θ (k) , θ (k) ) −H(θ (k+1) , θ (k) ) + H(θ (k) , θ (k) ) ≥ 0 (2) A remarkable result shows that the second line of (2) is non negative for any θ (k+1) (see McLachlan and Krishnan, 2008). Thus, we can guarantee that (2) holds if we guarantee that Q(θ (k+1) , θ (k) ) − Q(θ (k) , θ (k) ) ≥ 0. Camila Souza (UBC) Introduction to the EM algorithm November 2010 11 / 25
Page 1 and 2: Introduction to the EM algorithm Ca
Page 3 and 4: Example: Mixture of 2 normal distri
Page 5 and 6: In order to obtain the parameter es
Page 7 and 8: The EM algorithm The derivation of
Page 9: Note that log p(y|θ) = log p(y, z|
Page 13 and 14: Therefore, we can guarantee the ine
Page 15 and 16: Going back to our example Remember
Page 17 and 18: Since L 1 (λ’s) = n∑ M∑ I{z
Page 19 and 20: Since L 2 (α’s) = n∑ M∑ log
Page 21 and 22: M-sptep: Holding α j and σj 2 we
Page 23 and 24: In R I found two packages to work w
Page 25: Thank you !!! Camila Souza (UBC) In

Introduction to the EM algorithm - Department of Statistics

Create successful ePaper yourself

Delete template?

Save as template?