Nonparametric Bayesian Discrete Latent Variable Models for ...

4 Indian Buffet Process Models
matrix and taking the limit as the number of columns approaches infinity. Teh, Görür,
and Ghahramani (2007) construct a stick-breaking representation for the IBP, and show
interesting connections between the stick lengths of the stick-breaking construction for
the DP and the IBP. Thibaux and Jordan (2007) show that the underlying de Finetti
mixing distribution for the IBP is the beta process (Hjort, 1990). In the next section,
we present the description of the IBP and summarize these different approaches for
defining the distribution.
Analytical inference is not possible in the IBLF models. Due to the similarity of
the IBP and the DP, the methods for DPM models can be adjusted for the IBLF
models. Similar to the DPM case, Gibbs sampling in conjugate IBLF models is fairly
straightforward to implement but requiring conjugacy does not hold for many interesting
IBLF models. In Section 4.2, we summarize the Gibbs sampling algorithm for conjugate
IBLF models, as well as other MCMC algorithms that can be used for inference on the
non-conjugate IBLF models. In Section 4.3, we apply the described algorithms for
inference on a simple model on several synthetic data sets for empirically comparing the
performance of the different samplers.
In Section 4.4, we use an IBLF model suggested in (Griffiths and Ghahramani, 2005)
to learn the features of handwritten digits. We employ two-part latent vectors for
representing the hidden features of the observations: a binary vector to indicate the
presence or absence of a latent dimension and a real valued vector as the contribution
of each latent dimension. Using an IBP prior over the binary part, we have an overcomplete
dictionary of basis functions. In fact, we have infinitely many basis functions,
only a small number of which are active for each data point. We show that the model
can successfully reconstruct the images by finding features that make up the digits.
Another interesting application of IBLF is modeling the choice behavior using the
latent features to represent the alternatives in a choice set, Görür et al. (2006). In
Section 4.5 we describe the non-parametric choice model using IBP as a prior. We show
that the features that lead to the choice data can be inferred using the model.
There have been other interesting applications of IBP to define flexible latent feature
models. These include protein-protein interactions (Chu et al., 2006), the structure
of causal graphs (Wood et al., 2006b), dyadic data for collaborative filtering (Meeds
et al., 2007), human similarity judgements (Navarro and Griffiths, 2007) and document
classification (Thibaux and Jordan, 2007).
4.1 The Indian Buffet Process
The Indian buffet process (IBP) (Griffiths and Ghahramani, 2005) defines a distribution
on sparse binary matrices with infinitely many columns, and one row for each observation.
It is a sequential process that has been inspired by the Chinese restaurant process
(see Section 3.1.2) and is described by imagining an Indian buffet restaurant offering
an unbounded number of dishes. The metaphor is describing the N rows of a binary
matrix Z, each row represents a customer arriving at the restaurant and the infinitely
many columns of the matrix represent the dishes offered, a finite number of which will
68