Nonparametric Bayesian Discrete Latent Variable Models for ...

4 Indian Buffet Process Models
α µ zik
x (k) i
Figure 4.3: Graphical representation for the infinite latent feature model using the stick breaking
construction of the Indian buffet process. Note the strictly decreasing order of the
feature presence probabilities.
infinite matrix:
E{1 T Z T 1} = E
= K
ik
1
N
i=1
zik = K
0
= KN α/K
1 + α/K
8
N
E{zik}
i=1
µkp(µk)dµk
α
= N
1 + α/K
N
(4.14)
Taking the limit as K → ∞, the expected number of non-zero entries for the infinite
matrix is found to be αN, consistent with the result of the previous section.
The derivation presented above starts with defining the distribution over a finite
dimensional matrix and uses the conjugacy of the beta distribution to the binomial
distribution to integrate out the feature presence probabilities µk. Although the rows
are independent given µk, marginalization couples the rows, and the probability of an
entry is given in terms of the hyperparameter α and the number of other entries in that
column that are set to one, i.e. the number of objects sharing the feature. For the
distribution over the infinite matrix to be well defined, the column indices are ignored
by focusing on the permutation-invariant equivalence classes of matrices using the lof(·)
function before taking the infinite limit.
In the next section, we derive the same distribution with another approach, in which
the feature presence probabilities are explicitly represented instead of being integrated
out. To have a well defined distribution over the matrix with infinitely many columns,
a strictly decreasing ordering of the µks is imposed, which results in the so called stickbreaking
construction.
4.1.2 Stick-Breaking Construction
In Section 3.1.4 we described the stick-breaking construction for the DP. In this section,
we describe a similar construction for the IBP which allows representing the feature presence
probabilities explicitly in the IBLF model, see Figure 4.3. The derivation starts
74