Nonparametric Bayesian Discrete Latent Variable Models for ...

4.2 MCMC Sampling algorithms for IBLF models
Algorithm 12 Approximate Gibbs sampling for non-conjugate IBP
The state of the Markov chain consists of the infinite feature matrix Z and the set of
infinitely many parameters Θ = θ1:∞.
Only the K ‡ active columns of Z and the corresponding parameters are represented.
Repeatedly sample:
for all rows i = 1, . . . , N do {Feature updates}
for all columns k = 1, . . . , K ‡ do
if m−i,k > 0 then
Update zik by sampling from its conditional posterior, eq. (4.36).
else {m−i,k = 0}
Set one of the unallocated auxiliary parameters ˆ θj to θk.
end if
end for
Remove the columns of Z for which m−i,k = 0 and the corresponding parameters
θk from the representation.
Sample values from the prior G0 for the yet unallocated auxiliary parameters ˆ θj.
for all l = 1, . . . , 2 ˆ K do
Calculate the posterior for ˆ Zl using eq. (4.39)
end for
Pick a feature combination ˆ Zl with probabilities calculated above
Update Θ by appending ˆ Θ
Update Z by appending ˆ Zl
Remove zero columns of Z and the corresponding parameters θk from the representation.
Discard ˆ Zl and ˆ Θ
end for
for all active columns k = 1, . . . , K ‡ do {Parameter updates}
Update θk by sampling from its conditional posterior, eq. (4.35)
end for
87