Nonparametric Bayesian Discrete Latent Variable Models for ...

while for the empty features we have a Markov property:
p(µ ◦ (k) | µ◦ (k−1) ) ∝ (µ◦ (k) )α−1 (1 − µ ◦ (k) )N
4.3 Comparing Performances of the Samplers
exp( N i=1 1
i (1 − µ◦ (k) )i ))I{0 ≤ µ ◦ (k) ≤ µ◦ (k−1) }. (4.57)
Note that this equation is the same as eq. (4.51), with the conditioning on the rest of
Z being inactive is inherent in the definition of µ ◦ .
Slice Sampler
To use the semi-ordered stick-breaking construction as a representation for inference,
we can again use the slice sampler to adaptively truncate the representation for empty
features. This gives an inference scheme which works in the non-conjugate case, is
not approximate, has an adaptive truncation level, but without the restrictive ordering
constraint of the stick-breaking construction, summarized in Algorithm 16.
The representation consists only of the active features and the features and stick
lengths associated with these features. The slice variable is defined as
s ∼ Uniform[0, µ ∗ ] µ ∗
= min 1, min µ+
1≤k≤K ‡ k (4.58)
Once a slice value is drawn, we extend the representation by generating K◦ empty
features, with their stick lengths drawn from (4.57) until µ ◦ (K◦ +1) < s. The associated
feature columns Z◦ K◦ are initialized to 0 and the parameters θ◦ 1:K◦ are drawn from
their prior. Sampling for the matrix entries and the parameters proceed as before.
Afterwards, we remove the zero columns and the corresponding parameters and stick
lengths from the representation. Finally, the stick lengths for the new list of active
features are drawn from their conditionals (4.56).
We have presented several MCMC algorithms for inference on the IBLF models. The
important question is which of these algorithms to use in practice. In the next section,
we give an empirical comparison of some of the samplers.
4.3 Comparing Performances of the Samplers
We have described several sampling algorithms for inference on the models using IBP in
the previous section. It is important to have an intuition about the comparative performance
of the different samplers when choosing which one to use in practice. An especially
interesting question is how the computational cost is effected when non-conjugate
samplers are used. Therefore, we compare the mixing performance of the conjugate
Gibbs sampler (described in Algorithm 11) to the performance of the slice sampler using
the strictly decreasing ordering of the stick lengths (Algorithm 15) and using the
semi-ordered stick-breaking representation (Algorithm 16). We also compare the results
of the the approximate Gibbs sampler (Algorithm 12) to the conjugate Gibbs sampler
95