Nonparametric Bayesian Discrete Latent Variable Models for ...

∗
π ∗ π
s
3.2 MCMC Inference in Dirichlet Process Mixture Models
Figure 3.9: Slice sampling for DP. The vertical lines denote the mixing proportions πk of the
represented components in the stick breaking construction. Initially, there are six
components represented and for the data point being considered, ci = 4 (left). A
slice value s (shown by the dotted red line) is sampled from Uniform[0, π4] and
the represented components are extended until the sum of the remaining mixing
proportions is less than s (right).
strictly decreasing, but we know that they have to sum to 1. Therefore, if K † components
are represented, one of the unrepresented components can have a maximum stick
length of 1− K †
k=1 πk. If this value is greater than s, we keep breaking the stick until we
are left with a stick length smaller than s, see Figure 3.9 for a pictorial representation.
We sample parameters for the newly allocated components from the prior. Then we
update the indicator variable given the data and the components above the slice.
Algorithm 10 Slice sampling algorithm for the DPM model using the stick-breaking
construction.
The state of the Markov chain consists of indicator variables for each data point and
the infinitely many components with corresponding mixing proportions and parameters.
Only the mixing proportions and parameters of the components up to and including
the last active component are represented.
Repeatedly sample:
for all i = 1, . . . , N do {indicator updates}
Sample a slice variable s from the conditional distribution eq. (3.51)
if s < K †
l=1 πl then
Extend the representation by breaking the stick until s > K †
l=1 πl
Sample parameters for the new represented stick pieces from G0
end if
Assign xi to one of the components above the slice, using eq. (3.52)
end for
The methods presented in this section use the stick-breaking construction of the DP. In
this construction, the mixing proportions of the (infinitely many) mixture components
are represented. Therefore the indicator variables are updated without conditioning
on the other indicators. This feature of the samplers encourages good mixing for the
indicator variables. However, it should be noted that since the mixing proportions of
each component is represented explicitly, with a size-biased ordering of the cluster labels,
39