Nonparametric Bayesian Discrete Latent Variable Models for ...

4 Indian Buffet Process Models
Figure 4.10: Features that are shared between many images. Observe that different segments
are pronounced in different features.
effective latent dimension is determined by the number of nonzero columns of Z, similar
to the simpler model in the previous section. This model has two sets of parameters, Y
and A, associated with Z. We can not integrate over both of them. Therefore we need
an algorithm for non-conjugate IBP models for inference on this model. Motivated by
the results of the previous section, we choose to apply the semi-ordered slice sampler to
learn the latent features.
We modeled 1000 examples of the digit 3 in the MNIST data set. The digit images are
first preprocessed by projecting on to the first 64 PCA components, and the sampler is
ran for 10000 iterations. We show results for the digit 3. The distribution of the number
of nonzero features and the trace plots of the log likelihood are shown in Figure 4.12.
The model succesfully finds latent features to reconstruct the images as shown in
Figure 4.11. Some of the latent features found are shown in Figure 4.10. These appear
to model edge segments of the digit 3.
4.5 A Choice Model with Infinitely Many Latent Features
Modeling choice is an important topic of study for psychology, economics and related
sciences. The goal is to understand and model the behavioral process that leads to
the subject’s choice. In a choice scenario, the subject is presented with more than
one alternative from a choice set and is asked to choose one of the alternatives. The
alternatives may be items or courses of actions.
The answer to the question how much can be treated using regression models. The
choice models address the question which. Ordinal regression can be seen as a choice
model for which the alternatives have a particular ordering. Here, we will consider
discrete choice models where there is no particular ordering of the alternatives. Discrete
choice models assume there to be finitely many alternatives, all of which are included in
the choice set. The alternatives are assumed to be mutually exclusive, that is, choosing
one alternative implies not choosing any other. As data, we have outcomes of repeated
choice from subsets of the choice set. We use the number of times each alternative is
chosen over some others. We want to learn the true choice probabilities.
Psychologists have long been interested in the mechanisms underlying choice behavior
(Luce, 1959). In virtually all psychological experiments subjects are (repeatedly) asked
to make a choice and the responses are recorded. Often the choice is very simple like
100