Nonparametric Bayesian Discrete Latent Variable Models for ...

4 Indian Buffet Process Models
In the CRP, exchangeability is achieved by ignoring the labels of the tables and
focusing on the resulting partitioning. A similar approach can be taken for the matrices
by ignoring the column order. The analogue of partitions for class assignment vectors
are the equivalence classes for binary matrices. To establish a well defined distribution in
the infinite limit, Griffiths and Ghahramani (2005) define equivalence classes for binary
matrices with respect to a function called left-ordered form (lof (·)) and focus attention
on the distribution over the equivalence class of Z.
Defining the term history of a column of Z to refer to the binary number corresponding
to that column with the first row as the most significant bit, lof (Z) sorts the columns
of Z from left to right according to the histories. Any two binary matrices are lof -
equivalent if they map to the same left-ordered form. The lof -equivalence class [Z] of a
binary matrix Z is the set of binary matrices that are lof -equivalent to Z.
The columns of a matrix with N rows have (2N − 1) possible different histories (other
than zero, which corresponds to the column of zeros). Using the decimal equivalent
of the binary histories, Kh denotes the number of columns with history h. The total
number of columns of Z generated by the IBP can be expressed as K ‡ = 2N −1
h=1 Kh.
Considering the lof -equivalence classes of the matrices, there are
N i=1 K(i)
∗ !
2N −1
h=1 Kh!
matrices Z that can be generated by the sequential process that map to the equivalent
matrix [Z]. Therefore, the distribution over the equivalent feature matrices generated
by IBP becomes
P ([Z]) = αK‡
2N −1
Kh!
h=1
exp
− αHN
K‡
k=1
(N − m.,k)!(m.,k − 1)!
, (4.2)
N!
which accounts to both the customers and the dishes being exchangeable.
Griffiths and Ghahramani (2005) also describe the ”exchangeable Indian buffet process”
that directly produces matrices of the left-ordered form, which is established by
the customers attending to the histories of the dishes.
The process described has been inspired by the Chinese restaurant process. In the
CRP, customers choose which table to sit at, and favoring to be social, they tend to sit
at the more crowded tables. Since each customer can sit at only one table, the CRP
results in a partitioning on the customers. In the IBP, customers decide which dishes to
sample. They taste the more popular dishes with higher probability, and may also taste
some dishes which nobody tasted before. Every customer is free to sample any number
from the infinitely many dishes. Since customers can sample more than one dish, their
choices does not result in a partitioning, but can be seen as binary features that are
shared between data points.
Note that, in the CRP, although there are infinitely many tables available, the total
70