Nonparametric Bayesian Discrete Latent Variable Models for ...

4.5 A Choice Model with Infinitely Many Latent Features
Figure 4.11: Examples of reconstructed images and their latent features. Last column: original
digits, second last column: reconstructed digits, other columns: features used for
reconstruction. For each image, the binary vector zi determines the presence or
absence of a feature. The real valued vector zi determines how much each feature
contributes to the observed image.
log likelihood
m k (# objects with feature k)
×10 5
−4
−4.1
−4.2
−4.3
300
200
100
0
2500 5000
iterations
7500 10000
20 40 60 80 100 120
k (feature label)
# iterations
# objects
400
300
200
100
0
50 100
# active feats
150
150
100
50
0
5 10
# active feats
15
Figure 4.12: Distribution of the number of active features over 10000 iterations (left). Distribution
of number of active features for each input at one iteration (right). Note
that the number of features that a particular data point has is much less than
the total number of active features. The number of observations possessing each
feature at one iteration. Note that although the total number of active features is
much larger, about half of the features belong to only a few observations.
101