Nonparametric Bayesian Discrete Latent Variable Models for ...

4.2 MCMC Sampling algorithms for IBLF models
Algorithm 15 Slice sampling for stick-breaking IBP
The state of the Markov chain consists of the infinite feature matrix Z, the feature
presence probabilities µ (1:∞) = µ (1), µ (2), . . . corresponding to each feature column
and the set of infinitely many parameters Θ = θ1:∞.
Only the K † columns of Z up to and including the last active column and the corresponding
parameters are represented.
Repeatedly sample:
for all rows i = 1, . . . , N do {Feature updates}
Sample a slice s uniformly between 0 and the stick length of the last active component.
if s < µ (K † ) then
Extend the representation by breaking the stick until s > µ (K † ) using eq. (4.51)
Sample parameters for the new represented features from the prior
end if
for all columns k = 1, . . . , K † do {update features above the slice}
if µ (k) > s, update zik using the full conditional eq. (4.52)
end for
end for
for all columns k = 1, . . . , K † do {Parameter updates}
Update θk by sampling from its conditional posterior, eq. (4.35)
end for
for all columns k = 1, . . . , K † − 1 do {Update feature presence probabilities}
Update µk by sampling from its conditional posterior, eq. (4.53) using ARS
end for
for column K † , the last represented column, update µ (K † ), by sampling from its
conditional posterior, eq. (4.51)
4.2.6 Change of Representations and Slice Sampling for the Semi-Ordered
Stick-Breaking
Both the stick-breaking construction and the standard IBP representation are different
representations of the same nonparametric object. Therefore, it is possible to make use
of both representations in calculations by changing from one representation to the other
(Teh, Görür, and Ghahramani, 2007). More precisely, given a posterior sample in the
stick-breaking representation it is possible to construct a posterior sample in the IBP
representation and vice versa.
We use the infinite limit formulation of both representations to derive the appropriate
procedures. Note that for IBP, the feature labels in an arbitrarily large finite model are
ignored. On the other hand, the stick-breaking construction is obtained by explicitly
representing the feature presence probabilities and enforcing an ordering of the feature
indices with decreasing stick lengths. Therefore, for changing from the stick-breaking
to the IBP representation we simply drop the stick lengths as well as all the inactive
93