Nonparametric Bayesian Discrete Latent Variable Models for ...

4 Indian Buffet Process Models
features, leaving only the K ‡ active feature columns along with the corresponding parameters.
To change from IBP back to the stick-breaking representation, we have to
draw the stick lengths and sort the features in decreasing stick lengths, introducing
empty features if required.
We consider the finite binary matrix of Section 4.1.1 with the same distributional
assumptions. We index the K ‡ active features with k = 1, . . . , K ‡ . Let Z 1:K ‡ be the
feature presence matrix, that is, the matrix composed of only the active feature columns.
Suppose that we have K ≫ K ‡ features in the finite model. For the active features, the
posterior for the feature presence probabilities are simply
µ +
k | z:,k ∼ Beta( α
K + m·,k, 1 + N − m·,k), (4.54)
see eq. (4.5). Taking the limit as K → ∞, the posterior becomes,
Beta(m·,k, 1 + N − m·,k). (4.55)
In the stick breaking representation, we need to represent at least all features up to
and including the last active feature. Therefore, the representation may include some
inactive features. When changing the representation from IBP to the stick-breaking, it is
sufficient to represent only those empty features with stick lengths larger than mink µ +
k .
Thus we consider a decreasing ordering µ ◦ (1) > µ◦ (2) > · · · on the stick lengths of the
, we condition on the fact that there are no active
inactive components. For each µ ◦ (k)
features beyond that feature. Thus, considering infinitely many features, the density
for µ ◦ (k) is given by (4.51). ARS can be used to draw µ◦ (1:K ◦ ) until µ◦ (K ◦ ) < mink µ +
k .
We sample parameters for the newly represented inactive features from the prior. The
stick-breaking representation is obtained by reordering µ +
1:K ‡, µ ◦ (1:K◦ ) in decreasing order,
with the feature columns and parameters taking on the same ordering (columns and
parameters corresponding to empty features are set to 0 and drawn from their prior
respectively), resulting in K † = K ‡ + K ◦ features in the stick-breaking representation.
The validity of this representation can be seen by referring to the connection of the IBP
to the beta process Thibaux and Jordan (2007).
Semi-Ordered Stick-Breaking
In deriving the change of representations from the IBP to the stick-breaking representation,
we made use of an intermediate representation whereby the active features are
unordered, while the empty ones have an ordering of decreasing stick lengths. It is in
fact possible to directly work with this representation, which we shall call semi-ordered
stick-breaking.
The Z matrix consists of K ‡ active and unordered features, as well as an ordered
sequence of infinitely many empty features. The stick lengths for the active features
have conditional distributions:
94
µ +
k | z:,k ∼ Beta(m·,k, 1 + N − m·,k) (4.56)