Nonparametric Bayesian Discrete Latent Variable Models for ...

4 Indian Buffet Process Models
representation. We can sample entries of all columns from their conditional posterior
p(zik | µ (k), X, Θ, Φ) ∝ µ (k)F (X | zik = 1, Z−ik, X, Θ, Φ). (4.44)
Since the feature presence probabilities are explicitly represented in this construction,
they should also be updated. We obtain the posterior for µ (k) by combining the prior
from eq. (4.42) and the likelihood from eq. (4.43),
P (µ (k) | Z, µ −(k)) = α M µ α (M) µmk−1
(k) (1 − µ (k)) N−mk I{µ(k+1) ≤ µ (k) ≤ µ (k−1)} (4.45)
This density is not of standard form. But it can be shown that the log posterior is
log-concave, therefore it can be sampled from efficiently using ARS.
Algorithm 14 Gibbs sampling for truncated IBP
The state of the Markov chain consists of the finite feature matrix Z with M columns,
the feature presence probabilities µ1:M = µ1, . . . , µM corresponding to each feature
column and the set of parameters Θ = {θk} M 1 . All variables are represented.
Repeatedly sample:
for all rows i = 1, . . . , N and columns k=1,. . . ,M do {Feature updates}
Update zik by sampling from its conditional posterior, eq. (4.44).
end for
for all columns k = 1, . . . , M do {Parameter updates}
Update θk by sampling from its conditional posterior, eq. (4.35)
end for
for all columns k = 1, . . . , M do {Update feature presence probabilities}
Update µk by sampling from its conditional posterior, eq. (4.45) using ARS
end for
4.2.5 Slice Sampling Using the Stick Breaking Construction
Inference on the truncated stick-breaking construction using Gibbs sampling is easy
to implement and due to the ordering of the feature presence probabilities, the error
introduced by the truncation can be bounded. However, it is possible to avoid approximation
all together and do inference on the complete nonparametric model by using
slice sampling (Teh, Görür, and Ghahramani, 2007). This method can be interpreted
as adaptively choosing a truncation level at each iteration. See (Neal, 2003) for an
overview of slice sampling.
Slice sampling has been successfully applied to DP mixture models by Walker (2006)
(see Section 3.2.3), and the algorithm for IBLF models described below is related to
this approach.
In detail, we introduce an auxiliary slice variable,
90
s | Z, µ (1:∞) ∼ Uniform[0, µ ∗ ] (4.46)