Nonparametric Bayesian Discrete Latent Variable Models for ...

4.2 MCMC Sampling algorithms for IBLF models
as in the parametric models. Updating Z is more involved, since we have to deal with
the infinite dimensionality. Therefore, in the rest of this section, we will focus on updates
concerning Z.
In the following, we describe several different MCMC algorithms for inference on the
IBLF models. Conjugacy of the distribution of the feature parameters to the likelihood
function makes inference computationally easier. However, requiring conjugacy limits
the use of IBP. We start with describing sampling for conjugate IBLF models and
continue with other sampling algorithms that do not require conjugacy.
As for the DP, the sampling algorithms for IBLF models may be divided into two
subgroups: the ones that integrate out the feature presence probabilities µk, and the
ones that explicitly represent these probabilities using the stick-breaking construction.
We present algorithms that use these different approaches in the following subsections.
All methods described below update each entry of Z incrementally. Note that in the
IBP, the customers use different criteria for choosing dishes already sampled and for
choosing new dishes. As a consequence, the methods that use the representation where
µk is integrated out employ different update rules for the features owned by other data
points and for the (existing or new) unique features of the data point being considered.
On the other hand, for the methods that use the stick-breaking construction, we need
not make such a distinction.
In the following sections, Z denotes the binary matrix (with unbounded number of
columns), zik an entry of Z. The index i = 1, . . . , N runs over the rows, and k = 1, . . . , K
runs over the columns of Z. Similar to the terminology used in the previous chapter for
the DPM models, the columns of Z with non-zero entries will be referred to as the active
features, and the columns of zeros as the inactive features. Although Z has infinitely
many columns, only finitely many of them will be active. We denote the number of active
features with K ‡ and the number of features explicitly represented for the computations
(which includes all active features and possibly some inactive features) with K † . The
variable m is used to denote the number of entries that are set to 1 in a specified part
of Z, e.g. m.,k means number of 1s in the kth column, and m. 0 is proportional to the product of
the prior given in eq. (4.7) and the likelihood;
p(zik = 1 | Z−ik, X, Θ, Φ) ∝ m−i,k
N F (X | zik = 1, Z−ik, Θ, Φ). (4.36)
83