Nonparametric Bayesian Discrete Latent Variable Models for ...

4 Indian Buffet Process Models
column k in row i, given µ (k). This does not depend on i due to the exchangeability
of the rows. The distribution of the unordered feature presence probabilities µl after
the kth largest one is given by eq. (A.2). The entries zil are Bernoulli distributed with
probability µl. Marginalizing over µl, we have;
p(zil | µ (k)) =
=
µ(k)
0
µ(k)
0
p(zil | µl)p(µl | µ (k))dµl
α
= α
α + K µ (k).
µl
K µ−α/K
(k) µ α/K−1
l dµl
(4.28)
Taking the limit as K → ∞ of the Bernoulli trials with the above probability results in
a Poisson(αµ (k)) distribution over the number of non-zero entries in the ith row. This
result is intuitive, stating that the expected number of non-zero entries depends on the
limiting µ (k) value, and the parameter α. If we consider the entries of the whole row,
i.e. the case k = 0, we recover the distribution derived in the previous section for the
total number of non-zero entries in a row, Poisson(α).
We can also calculate the probability of all entries to the right of column k being zero:
p(Z (:,.>k) = 0 | µ (k)) = exp − αHN + α
N
i=1
(1 − µ (k)) i
i
, (4.29)
where HN is the Nth harmonic number. See Appendix A for the details of derivation.
4.1.3 A Special Case of the Beta Process
Beta process has been defined for use in survival analysis as a cumulative hazard rate
process with nonnegative independent increments 2 by Hjort (1990). Thibaux and Jordan
(2007) show that a special case of the beta process is related to the Indian buffet process
(IBP) the way the Dirichlet process is related to the Chinese restaurant process.
They define the following generative model for the binary feature matrix Z:
A ∼ BP(c, αA0),
Z ∼ BeP(A),
(4.30)
where BeP(A) denotes a Bernoulli process with hazard measure A and BP(c, αA0)
denotes a beta process with base measure αA0 and concentration function c. A0 is a
probability measure, α a positive scalar, and c a deterministic function. The distribution
of Z is equivalent to that of the matrix generated by the IBP(α) for the particular choice
of c being the constant function c = 1, BP(1, αA0) .
Considering the beta process BP(c, αA0) with a constant concentration function in
the above generative model results in a two-parameter generalization of the Indian buffet
2 See Appendix B.3 for the definition of the independent increment processes
78