Nonparametric Bayesian Discrete Latent Variable Models for ...

4 Indian Buffet Process Models
The cumulative distribution function of the maxima of a set of independent random
variables is the product of the cumulative distribution functions of each of the variables.
We obtain the cdf for µ (1) by taking the product of the K (identical) cdf’s,
F (µ (1)) = µ α
K
(1) I{0 ≤ µ (1) ≤ 1} + I{1 < µ (1)} K
= µ α (1) I{0 ≤ µ (1) ≤ 1} + I{1 < µ (1)}.
Differentiating, we obtain the probability density function (pdf) of µ (1):
That is, µ (1) ∼ Beta(α, 1).
(4.20)
p(µ (1)) = αµ α−1
(1) I{0 ≤ µ (1) ≤ 1}. (4.21)
Following the same approach for the subsequent feature presence probabilities, the
density of µ (k+1) is obtained to be,
p(µ (k+1) | µ (1:k)) = p(µ (k+1) | µ (k))
= α
K − k
K
K−k
K
µ−α
(k)
µ α K−k
K −1
(k+1)
I{0 ≤ µ (k+1) ≤ µ (k)}.
(4.22)
This is the density of the (k + 1)th largest value of K random variables all with distribution
given in eq. (4.16). To obtain the distribution of the probabilities corresponding
to the columns of the infinite matrix, we take the limit as K → ∞. In the limit, the
density of µ (k+1) becomes
p(µ (k+1) | µ (k)) = αµ −α
(k) µα−1
(k+1) I{0 ≤ µ (k+1) ≤ µ (k)}. (4.23)
Defining µ (0) = 1, the above equation gives the densities for the feature presence probabilities
for all columns of the infinite dimensional binary matrix Z sorted in a strictly
decreasing order.
Note that given µ (k), µ (k+1) is independent of all other µ values. We introduce a
set of variables νk = µ (k)
µ (k−1) to make use of this Markov property. Since µ (k) has range
[0, µ (k−1)], νk has range [0, 1]. Using a change of variables, the distribution of νk can be
obtained from eq. (4.23) to be,
p(νk | µ (k−1)) = p(µ (k) | µ (1:k−1))
dµ
(k)
dνk
= αν α−1
k I{0 ≤ νk ≤ 1}.
(4.24)
Thus, νk is independent from µ (1:k−1) and is simply Beta(α, 1) distributed. We obtain
the stick-breaking representation by expanding µ (k),
76
µ (k) = νk µ (k−1) =
k
νl. (4.25)
l=1