Nonparametric Bayesian Discrete Latent Variable Models for ...

3.2.2 Algorithms for non-Conjugate DP Models
No Gaps Algorithm
3.2 MCMC Inference in Dirichlet Process Mixture Models
The algorithms described above use the indicator variables ci, i = 1, . . . , N to assign
identical values of θ to the component parameters φ. The set of numerical values of
ci do not have significance beyond denoting the grouping. The ”no gaps” algorithm of
MacEachern and Müller (1998) constrains the labels of the active components to cover
the integers from 1 to K ‡ , and augments the state to include empty components so as
to have a total of N represented components, i.e. K † = N. Augmenting the model
replaces the integral evaluations with likelihood evaluations.
The state of the Markov chain consists of the indicator variables and the parameters
of the N components, K ‡ of which have data assigned, the rest being empty. First the
to denote the
number of distinct groups of data not including the ith data point. Label the groups
indicator variable of each data point i is updated as follows. Use K ‡
−
from 1 to K ‡
− . If ci is a singleton, with probability K ‡
− /(K‡ − + 1) leave ci unchanged,
otherwise label ci as (K ‡
− + 1), consequently assigning φK ‡ to be the existing value
− +1
for φci . Update ci with the following probabilities:
P (ci = k | xi, c−i, φ) ∝ n−i,kF (xi | φk) for k = 1, . . . , K ‡
− ,
P (ci = K ‡
− + 1 | xi, c−i, φ) ∝
α
K ‡
− + 1F (xi | φ K ‡
− +1)
(3.33)
After the indicator updates, update the component parameters by sampling from their
conditional posterior, eq. (3.31).
Algorithm 4 The No-Gaps algorithm for non-conjugate DPM models.
The state of the Markov chain consists of the indicator variables c = {c1, . . . , cN} and
N component parameters Φ = {φ1, . . . , φN}
Of the N parameters, only K ‡ + 1 of them are represented
Repeatedly sample:
for all i = 1, . . . , N do {indicator updates}
denote the number of active components without considering xi
Let K ‡
−
if ci is a singleton then
With probability K ‡
− /(K‡ − + 1) do not update ci,
otherwise, label ci as K ‡
−
end if
+ 1
Label the components of all data points other than i from 1 to K ‡
−
Update ci using eq. (3.33)
end for
for all k = 1, . . . , N do {parameter updates}
Update φk by sampling from its posterior, eq. (3.31)
end for
29