Nonparametric Bayesian Discrete Latent Variable Models for ...

4.2 MCMC Sampling algorithms for IBLF models
Algorithm 13 Metropolis-Hastings sampling for IBP
The state of the Markov chain consists of the infinite feature matrix Z and the set of
infinitely many parameters Θ = {θk} ∞ 1 .
Only the K ‡ active columns of Z and the corresponding parameters are represented.
Repeatedly sample:
for all rows i = 1, . . . , N do {Feature updates}
for all columns k = 1, . . . , K ‡ do
if m−i,k > 0 then
Update zik by sampling from its conditional posterior, eq. (4.36).
end if
end for
Propose a number ˆ K for unique features from the prior Poisson(α/N)
Sample ˆ K parameters for the unique features. and Θ ˆ K
Evaluate the proposal using the acceptance ratio, eq. (4.40)
end for
for all active columns k = 1, . . . , K ‡ do {Parameter updates}
Update θk by sampling from its conditional posterior, eq. (4.35)
end for
tial rate, which suggests adapting the truncation used for approximating the DP to the
IBP. The bound on the error introduced by the truncation for the DP stick-breaking
construction has been calculated by Ishwaran and James (2001). Noting the correspondences
between the stick weights of the IBP and the DP, a similar approach can be used
in this case.
Let M be the truncation level. Setting µ (M+1) = 0 constrains all µ (k) = 0 for k > M,
while the joint density for µ (1:M) is given as
p(µ (1:M)) =
M
k=1
p(µ (k) | µ (k−1))
M
=α M µ α (M)
k=1
µ −1
(k) I{0 ≤ µ (M) ≤ · · · ≤ µ (1) ≤ 1}
(4.42)
Inference using Gibbs sampling is straightforward to implement on the truncated
model. The entries of Z are independent given µ (1:M), thus
p(Z | µ (1:M)) =
N
M
µ
i=1 k=1
zik
(k) (1 − µ (k)) 1−zik . (4.43)
Since the entries in a column are independent given the feature presence probabilities,
we do not need to worry about whether the other data points have the feature being
updated or not. That is, we do not need separate update rules for separate cases in this
89