Nonparametric Bayesian Discrete Latent Variable Models for ...

3 Dirichlet Process Mixture Models
current duration
Old Faithful data set
previous duration
conjugate model
conditionally conjugate model
kernel density estimation
Figure 3.12: The Old Faithful geyser data set and its density modelled by CDP, CCDP and
KDE. The two dimensional data consists of the durations of the consecutive eruptions
of the Old Faithful geyser.
As a measure for modeling performance, we use the average leave one out predictive
densities. That is, for all data sets considered, we leave out one observation, model the
density using all others, and calculate the predictive density on the left-out data point.
We repeat this for all data points in the training set and report the average predictive
density.
The mixing of all samplers is equally fast for the two dimensional Geyser data set.
There is also not a significant difference in the predictive performance, see Tables 3.1
and 3.2. However, we can see form the plots in Figure 3.12 that the resulting density
estimates are different for all models.
For all data sets, the KDE model has the lowest average leave one out predictive
density, and the conditionally conjugate model has the best (see Table 3.1). To compare
the distribution of the leave one out densities, p-values for a paired t-test are given
(Table 3.2). For the Spiral, Iris and Wine data sets, the difference between the predictive
densities of KDE and both DP models were statistically significant.
The main objective of the models presented in this paper is density estimation, but
the models can be used for clustering as well by observing the assignment of data points
to model components. Since the number of components change over the chain, one
48