Nonparametric Bayesian Discrete Latent Variable Models for ...

4.1 The Indian Buffet Process
from the distribution on a finite binary matrix and imposing a strictly decreasing ordering
of the feature presence probabilities before taking the infinite limit. Denoting the
kth largest feature presence probability with µ (k), we will show that the stick-breaking
construction for the IBP has the following law for the feature presence probabilities,
µ (k) = νk µ (k−1) =
k
νl, with µ (0) = 1 and νk ∼ Beta(α, 1). (4.15)
l=1
That is, each feature presence probability µ (k) is a product of the previous one, µ (k−1),
and the random variable νk. This can be described metaphorically as starting with
a stick of unit length (µ (0) = 1), and breaking a piece off the stick at each iteration,
discarding that piece and recursing on the piece that we kept. The breaking point is
determined by the variable νk, and the feature presence probability µ (k) is the stick
length left after k iterations. Note that this procedure enforces a strictly decreasing
ordering of the µ’s since µ (k) is given by the stick length that we have left after the kth
iteration 1 and we recurse on the piece of the stick left at each iteration, see Figure 4.4
for a pictorial representation.
In the following, we present the outline of derivation of the stick-breaking construction
for the IBP. Details of the derivation are given in Appendix A. We start with the same
distributional assumptions on the entries of a finite binary matrix with N rows and K
columns as in Section 4.1.1. That is, for the unordered columns we assume
The density of µk is given as:
p(µk) =
(zik | µk) ∼Bernoulli(µk)
µk ∼Beta( α
, 1).
K
α Γ( K + 1)
Γ( α
K
α
)Γ(1)µ k
K −1
= α α
µ K
K −1 I{0 ≤ µk ≤ 1},
(1 − µk) 1−1 I{0 ≤ µk ≤ 1}
(4.16)
(4.17)
where I{A} is the indicator function for a measurable set A; I{A} = 1 if A is true, 0
otherwise. The cumulative distribution function (cdf) for µk is:
F (µk) =
µk α α
t K
0 K −1 I{0 ≤ t ≤ 1}dt
= µ α
K
k I{0 ≤ µk ≤ 1} + I{1 < µk}.
(4.18)
We define µ (1) ≥ µ (2) ≥ · · · ≥ µ (K) to be the decreasing ordering of µ1, . . . , µK. Thus,
µ (1) is defined as
µ (1) = max
k=1,...,K µk, (4.19)
1 For the DP, the mixing proportions correspond to the length of the piece that we discard. This
relation be discussed further in the section
75