Nonparametric Bayesian Discrete Latent Variable Models for ...

ch 2
ch 3
ch 1
ch 2
ch 3
ch 4
3.4 Dirichlet Process Mixtures of Factor Analyzers
ch 1
ch 2
Figure 3.22: Peak-to-peak amplitude plots showing the clustering results of DPMFA using the
whole waveforms. Note that one big cluster of manual clustering (light blue) is
divided into three clusters with this model (light blue, brown and purple).
Gibbs sampling.
The behavior of the Markov chain for the DPMFA model on the PCA components and
the whole waveforms is interesting. No matter if initialized with a single component or
many components, the model first explores the space by introducing many components
and eventually reduces the number of active components to give a good representation of
the data. The sampling can handle this for the lower dimensional PCA inputs, however
for the 112 dimensional inputs, it fails to converge to a stationary distribution when all
variables are updated. For this reason, for using the waveforms as inputs, we fixed the
concentration parameter α to 1. An alternative can be to have better initial values for
the hyperparameters but this would involve a thorough analysis of the data.
The motivation for developing this model is to be able to apply the DPM model to high
dimensional data. Experimental results show that the sampling algorithms used are not
efficient enough. Using the split-merge method of Jain and Neal (2005) would speedup
mixing. Furthermore, the factor loading matrix is rotation independent, therefore not
unique. Restricting it to be unique, e.g. forcing it to be lower diagonal as suggested by
Fokoué and Titterington (2003) may also help speedup mixing.
ch 4
ch 4
ch 1
ch 3
63