Online Model Selection Based on the Variational Bayes

<strong>Online</strong> <strong>Model</strong> <strong>Selection</strong> <strong>Based</strong> on the Variational Bayes 1671
Figure 4: <strong>Online</strong> model selection processes using the online VB method for
data set B. Free energies (FE) and number of units for the base model (solid
line), which is the best model to date, and the current trial model (dash-dotted
line) are shown. Log likelihood (LKD) for the current trial model (dashed line)
is also shown. (A) The initial model consists of 2 units. (B) The initial model
consists of 10 units.
�fty-�rst epoch, the number of gaussians was six. In the learning process,
each model observed 1000 data in one epoch. Figure 7 shows the learning
process. After the twentieth epoch, the optimal model structure with four
units was found. When the environment was changed at the �fty-�rst epoch,
the free energies of both models dropped dramatically. The trained models
tried to search for the new optimal structure. After the seventy-�fth epoch,
the optimal model structure with six units was found. The result showed
that the online VB method was able to adjust the model structure to dynamic
environment.