Online Model Selection Based on the Variational Bayes

<strong>Online</strong> <strong>Model</strong> <strong>Selection</strong> <strong>Based</strong> on the Variational Bayes 1655
conjugate distribution for the EFH model with posterior hyperparameters
(®, c ) (see appendix A):
Q h (µ ) D P a (µ | ®, c ) D exp [c (® ¢ µ ¡ Ã (µ )) ¡ © (®, c )] , (2.16)
c D T C c 0, (2.17)
® D 1 h i
Thr(x, z)i N C ®0 ¢ c 0 ,
c µ
(2.18)
hr(x, z)i N D
µ 1
TX
Z
T
dm (z(t))P(z(t)|x(t), µ N )r(x(t), z(t)). (2.19)
tD1
The effective amount of data c D (T C c 0) represents the reliability (or uncertainty)
of the estimation. As the amount of data T increases, the reliability
of the estimation increases. The prior hyperparameter c 0 represents the reliability
of the prior belief on the prior hyperparameter ®0. The posterior
hyperparameter ® is determined by the expectation value of the suf�cient
statistics. The prior hyperparameter ®0 gives the initial value for ®.
Since the posterior parameter distribution Qh (µ ) is given by the conjugate
distribution Pa (µ | ®, c ), which is also an exponential family model, the
integration over the parameter µ in equation 2.15 can be explicitly calculated
as
Nµ D hµi ® , (2.20)
Z
hµi ® D dm (µ )Pa (µ | ®, c )µ D 1 @©
c @® (®, c ). (2.21)
The natural parameter of the conjugate distribution is given by (c ®, c ). The
corresponding expectation parameters are given by the ensemble averages
of the model parameters: hµi ® de�ned in equation 2.21 and
hà (µ )i ® D
Z
D 1
c
dm (µ )Pa (µ | ®, c )Ã (µ )
@©
@® (®, c ) ¢ ® ¡ @©
@c (®, c ). (2.22)
2.5 Parameterized Free Energy Function. Since the optimal solution simultaneously
satis�es equations 2.14 and 2.16, the trial posterior distributions,
Q h (µ ) and Qz(ZfTg), can be parameterized as
Qh (µ ) D Pa (µ | ®, c ) D exp [c (® ¢ µ ¡ Ã (µ )) ¡ © (®, c )] , (2.23)
TY
Qz(ZfTg) D Qz(z(t)), (2.24)
tD1
Qz(z(t)) D P(z(t)|x(t), N µ ), (2.25)