Nonparametric Bayesian Discrete Latent Variable Models for ...

4 Indian Buffet Process Models
Algorithm 16 Slice sampling for the semi-ordered IBP
The state of the Markov chain consists of the infinite feature matrix Z, the feature
presence probabilities µ1:∞ = µ1, . . . , µ∞ corresponding to each feature column and
the set of infinitely many parameters Θ = {θk} ∞ 1 .
Only the K ‡ active columns of Z up to and including the last active column and the
corresponding parameters are represented.
Repeatedly sample:
Change to SB representation:
Sample µs for active features (µ + ) from their posterior, eq. (4.56)
Sample µs for inactive components (µ ◦ ) using eq. (4.57) until the smallest µ + is larger
than the smallest µ ◦
Sort columns to have µs in decreasing order
for all i = 1, . . . , N do
Do feature updates in the SB representation
end for
Change to IBP representation:
Remove feature presence probabilities from the representation
Remove inactive feature columns from the representation
for all columns k = 1, . . . , K † do {Parameter updates}
Update θk by sampling from its conditional posterior, eq. (4.35)
end for
results to have a sense of the accuracy of the approximation. Since for both cases the
conjugate Gibbs sampling is taken as a basis of comparison, we use a conjugate model.
We choose to use the linear-Gaussian binary latent feature model from Griffiths and
Ghahramani (2005). The model is summarized below, see the referred paper for a
detailed description. Each data point xi is assumed to be generated by a combination
of a subset of the rows of A and distorted by spherical Gaussian noise,
xi = ziA + ε, (4.59)
with ε ∼ N (0, σ 2 xI). The infinite dimensional binary vector zi encodes which features
are contributing to xi, and A is a matrix (with infinitely many rows) whose kth row
corresponds to the parameters for the kth feature. This model can be interpreted as a
binary factor analyzer with an infinite dimensional latent space. The distribution over
the whole data matrix X can be written as a matrix Gaussian,
X | Z, A, σx ∼ N (ZA, σ 2 xI). (4.60)
Entries of A are drawn i.i.d. from a zero-mean Gaussian with variance σ2 A . We can
96