Nonparametric Bayesian Discrete Latent Variable Models for ...

and Bush and MacEachern (1996).
3.2 MCMC Inference in Dirichlet Process Mixture Models
In detail, the state of the Markov chain consists of the indicator variables c =
{c1, . . . , cN} and the component parameters φc = {φk, k ∈ c} The prior for the indicator
variables is given by eq. (3.8). Combining this with the likelihood, the conditional
posterior is;
assigning to components for which n−i,k > 0 :
assigning to a new component:
P (ci = k|xi, c−i, α, φ) ∝
P (ci = ci ′ for all i = i′ |xi, c−i, α) ∝
n−i,k
N − 1 + α F (xi | φk),
α
N − 1 + α
F (xi | φ)dG0(φ).
(3.30)
That is, we either choose to assign the data point i to one of the existing components, or
, defined in eq. (3.28).
create a new active component by sampling its parameter from Gi 0
Note that if xi belongs to a singleton component, the parameter value of that component
will be removed from the representation when ci is updated. After updating the indicator
variables, the component parameters are updated by sampling from their posterior
conditioned on the data points assigned to them,
P (φk | X, c) ∝ F (xi, ∀ci = k | φk)G0. (3.31)
Algorithm 2 Gibbs sampling for conjugate DPM models using indicator variables and
component parameters.
The state of the Markov chain consists of the indicator variables c = {c1, . . . , cN} and
the component parameters φc = {φk, k ∈ c}
Repeatedly sample:
for all i = 1, . . . , N do
Update ci using eq. (3.30)
If ci is assigned to a singleton, sample a parameter for this component from G0
end for
for all k = 1, . . . , K † do
Update φk by sampling from its posterior, eq. (3.31)
end for
This algorithm can be further simplified by integrating over the φ, and eliminating
them from the state (Neal, 1992; MacEachern, 1994).
27