Nonparametric Bayesian Discrete Latent Variable Models for ...

4.2 MCMC Sampling algorithms for IBLF models
∗ µ ∗
µ
Figure 4.6: Pictorial representation of slice sampling for IBP. The slice value is sampled uniformly
randomly between 0 and the stick length of the last active feature. The slice
is cut through the stick pieces. The representation is extended to include all features
with stick lengths higher than the slice.
where µ ∗ is a function of µ (1:∞) = {µ (k)} ∞ 1 and Z, and is chosen to be the length of the
stick for the last feature with a non-zero entry,
µ ∗
= min 1, min
k: ∃i,zik=1 µ
(k) . (4.47)
The joint distribution of Z and the auxiliary variable s is
where
s
p(Z, s | x, µ (1:∞)) = p(Z | x, µ (1:∞)) p(s | Z, µ (1:∞)) (4.48)
p(s | Z, µ (1:∞)) = 1
µ ∗ I{0 ≤ s ≤ µ∗ }. (4.49)
Clearly, integrating out s preserves the original distribution over Z, while conditioned
on Z and µ (1:∞), s is simply drawn from (4.46). Given s, the distribution of Z becomes:
p(Z | x, s, µ (1:∞)) ∝ p(Z | x, µ (1:∞)) 1
µ ∗ I{s ≤ µ ∗ } (4.50)
which forces all columns k of Z for which µ (k) < s to be zero. This can be interpreted
as the auxiliary variable s cutting a slice through the feature presence probabilities,
leaving only those with a larger weight than s to be updated. Since s is sampled
uniformly between 0 and the height of the last active feature, the features below the
slice are fixed to zero.
Let ˜ K be the maximal feature index with µ ( ˜ K) > s. Thus zik = 0 for all k > ˜ K,
and we need only consider updating those features k ≤ ˜ K. Notice that ˜ K serves as a
truncation level insofar as it limits the computational costs to a finite amount without
approximation.
Let K † be an index such that all active features have index k < K † (note that K † itself
would be an empty feature). The computational representation for the slice sampler consists
of the slice variables and the first K † features: {s, ˜ K, K † , Z 1:N,1:K †, µ (1:K † ), θ 1:K †}.
The slice sampler proceeds by updating all variables in turn.
The slice variable is drawn from (4.46). If the value of s is less than the last represented
feature presence probability µ (K † ), then we need to extend the representation. We
91