Nonparametric Bayesian Discrete Latent Variable Models for ...

4.2 MCMC Sampling algorithms for IBLF models
the beta process is that a draw from a beta process has the form ∞
(·) with
k=1 µ (k)δθk
µ (k) drawn form the stick breaking prior for the feature presence probabilities, and θ (k)
is independent from µ (k) and is drawn from the base measure H. Generalizations of the
stick-breaking constructions lead to generalizations of the beta process.
The IBP parameter α affects the number of active features therefore updating this
parameter would give more flexibility to the model. The likelihood for α can be derived
from the joint distribution of the features given in Equation 34 of Griffiths and
Ghahramani (2005) to be,
p(Z | α) ∝ α K‡
exp
− αHN ,
where K ‡ is the number of active components and N is the number of rows. We can
put a gamma prior on α,
α ∼ G(1, 1).
Combining this likelihood with the prior, we get the posterior distribution for α
p(α | Z) = G 1 + K ‡
, 1 + HN . (4.33)
Since this posterior is of standard form, it can be sampled from easily.
Note that the construction of IBP and related distributions described above concerns
the prior for Z only. Using Z in a IBLF, we update it by sampling its entries using the
full conditional distributions which involves conditioning on the data. In the following
section, we describe methods for inference on IBLF models.
4.2 MCMC Sampling algorithms for IBLF models
The previous section summarized several different approaches for defining the distribution
induced by the Indian buffet process, which has strong connections to the Dirichlet
Process. In this section, we describe MCMC methods for inference on the latent feature
models that use the IBP as the nonparametric prior over the feature matrix, which we
refer to as the infinite binary latent feature (IBLF) models.
The general form of the models we will consider assumes the data generating process to
be living in a latent space of possibly infinite dimension. Each data point xi is described
by an infinite dimensional binary latent vector zi = zi,1:∞ that encodes the presence
or absence of the infinitely many features characterized by the parameters Θ = θ1:∞,
and possibly other parameters Φ that live in the observable space. The state of the
binary latent variables determine which features are actively used, that is, the effective
latent dimensionality. Equivalently, we refer to the vector zi as the binary latent feature
vector, and Θ as the set of parameters of the features. We put an IBP prior on the
feature presence matrix Z, which has the vectors zi as its rows. The parameters Θ and
Φ are given standard parametric priors. The data distribution is assumed not to depend
on the features that do not belong to any of the data points, that is, the zero columns
of Z and the parameters θk that are associated with those columns. The model can be
81