Nonparametric Bayesian Discrete Latent Variable Models for ...

3.2 MCMC Inference in Dirichlet Process Mixture Models
ratio of changing the component assignment ci from k to j is given as:
⎧
1, if j ≤ K † and K † = K †
kj
M(K †
kj )
, if j ≤ K † and K †
kj < K†
⎪⎨ min 1,
α =
κ(K† )
κ(K †
kj )
⎪⎩
min 1, κ(K† )
κ(K †
kj )
F (xi | θk)
F (xi | θj)
M(K †
, if j > K
)
†
(3.47)
where we used K †
kj to denote the index of the last active component after changing
component assignment of xi form k to j. Thus, the algorithm behaves like a Gibbs
sampler when the assignment to an already represented component is proposed if this
move does not change last active component. This algorithm provides a sampler that
does not need to resort to approximations. However, the choice of Mi should be made
carefully to satisfy detailed balance when the index for the last active component changes
and obtain a good mixing performance.
Algorithm 9 Retrospective sampling for DPM models using stick-breaking construction
The state of the Markov chain consists of the stick lengths (mixing proportions) and
the component parameters.
Only the mixing proportions and parameters of the components up to and including
the last active component are represented.
Repeatedly sample:
for all i = 1, . . . , N do
Sample ui ∼ Uniform[0, 1]
Evaluate the posterior probabilities for component assignments using eq. (3.45).
if k
l=1 P (ci = l) > ui for k ≤ K † then
Represent more components by breaking the stick to obtain the mixing propor-
tions and sampling parameters from the prior until k
l=1 P (ci = l) < ui for
l ≤ K † .
end if
Propose to move to ci = inf{k ≤ K † : k
l=1 P (ci = l) < ui}
Evaluate the proposal with eq. (3.47)
end for
Slice Sampling
The idea of auxiliary variable methods is sampling for the variables we are interested in
by using auxiliary variables which make the updates easier to handle. Slice sampling is
an auxiliary variable method that exploits the fact that sampling from a distribution is
equivalent to sampling uniformly from the region under its probability density function
37