Nonparametric Bayesian Discrete Latent Variable Models for ...

4 Indian Buffet Process Models
z
α ξ γ
ik
x i
θ ik
Figure 4.5: Graphical representations of a general latent feature model. The data xi is generated
by latent features, zi, parameters associated with the latent features, θk and other
parameters φ, all of which have assigned prior distributions. The hierarchy can be
extended by specifying priors on the hyperparameters.
summarized as follows:
8
N
(xi | zi, Θ, Φ) ∼ F (xi | zi, Θ, Φ)
Z ∼ IBP (α)
θk ∼ G0(θk | ξ)
Φ ∼ H(Φ | γ)
Φ
(4.34)
where F (·) is the distribution of the data and G0 and H are prior distributions for the
parameters, specified by hyperparameters. See Figure 4.5 for a graphical representation.
The model can be extended by adding more layers to the hierarchy by specifying priors
for the hyperparameters. Note that there are infinitely many latent features. But since
the data distribution does not depend on the inactive features, inference on this model
is still tractable.
We will consider updating each set of variables sequentially. Given the nonparametric
part of the model concerning the latent variables, i.e. given Z and Θ, the updates
for the rest of the parameters Φ is just like in the parametric hierarchical models.
Furthermore, although Θ is infinite dimensional, Z will only have finitely many nonzero
columns. Since the likelihood does not depend on the zero columns, the posterior
for θk corresponding to the inactive columns of Z is same as the prior. Therefore, we
only need to update those θk that are associated with the active features. This is again
simply sampling from the conditional posterior of θk,
82
P (θk | X, Z, Φ) ∝ G0(θk | ξ)F (X | Z, Θ, Φ) (4.35)