Online Model Selection Based on the Variational Bayes

<strong>Online</strong> <strong>Model</strong> <strong>Selection</strong> <strong>Based</strong> on the Variational Bayes 1677
Yh (!,
"
MX
#
µ ) D log exp(vi C Yi(µi)) . (C.5)
iD1
The conjugate distribution for the MEF model, equation C.3, is given by the
product of the Dirichlet distribution and the conjugate distribution for each
unit:
P a (g, µ | ®, º, c ) D exp
Wa (®, º, c ) D
D exp
"
"
c
c
MX
¡ ¢
ºi log gi ¡ Yi(µi) C ®i ¢ µi ¡ Wa (®, º, c )
iD1
MX
(ºivi C ºi®i ¢ µi)
iD1
¡c Yh (!, µ ) ¡ Wa(®, º, c )
MX
log C (c ºi C 1)
iD1
¡ log C (c C M) C
#
#
, (C.6)
MX
Wi(®i, c ºi), (C.7)
where ºi satis�es PM
iD1 ºi D 1, and C (c ) is the gamma function, that is,
C (c ) D R 1
0 dse¡s s c ¡1 . The VB algorithm for the MEF model can be derived
by using Wa as described in section 2. The VB E-step equation is given by
Nµi D hµii ® D 1
c ºi
Nvi D hvii ® D 1
c
@Wa
@®i
@Wa
@ºi
The VB M-step equation is given by
iD1
, (C.8)
¡ ®i ¢ hµii ® . (C.9)
c D T C c (0), (C.10)
ºi D 1 ±
Thzii N C c (0)ºi(0)
c µ ² , (C.11)
®i D 1 ±
Thziri(x)i N C c (0)ºi(0)®i(0)
c ºi µ ² , (C.12)
where c (0) and fºi(0), ®i(0)|i D 1, . . . , Mg are the prior hyperparameters
of the prior parameter distribution. h¢i N µ denotes the expectation value (see
equation 2.19) with respect to P(z|x, N!, N µ ).