Nonparametric Bayesian Discrete Latent Variable Models for ...

4.5 A Choice Model with Infinitely Many Latent Features
See Figure 4.13 for a graphical representation of the model.
4.5.2 Inference using MCMC
Inference for the above model can be done using MCMC techniques. Görür et al. (2006)
present results using the approximate Gibbs sampling for updating the feature matrix
Z. Here, we use slice sampling with the semi-ordered stick-breaking representation. We
use Gibbs sampling for the IBP parameter α and Metropolis Hastings updates for the
weights w.
Gibbs sampling for the feature updates requires the posterior of each zik conditioned
on all other features Z −(ik) and the weights w. The conditional posterior for the entries
of the feature matrix can be obtained by combining the likelihood given in eq. (4.65)
with the prior feature presence probability µ (k),
P (zik = 1 | Z−ik, w, µ (k)) ∝ µ (k)P (X | zik = 1, Z−ik, w, µ (k)). (4.68)
We update the weights using Metropolis Hastings sampling. We sample a new weight
from a proposal distribution Q(w ′ k |wk) and accept the new weight with probability
min
1, P w ′ k |X, Z, w−k, wk, λ
P wk|X, Z, w−k, w ′ k , λ Q(wk|w ′ k )
Q(w ′ k |wk)
. (4.69)
As the proposal distribution we use a gamma distribution with mean equal to the current
value of the weight, wk, and standard deviation proportional to it,
Q(w ′ k |wk) = G(ηwk, η/wk). (4.70)
We adjust η to have an acceptance rate around 0.5 initially.
Note that there are infinitely many weights that are associated with the infinitely
many features. Since the inactive features and their weights do not affect the likelihood,
we need to only represent and update the weights that are associated with the active
features.
4.5.3 Experiments
In this section, we present empirical results on an artificial and a real data set. Both
data sets have been considered in the choice model literature.
We initialize the parameters α, Z and w randomly from their priors and set the lapse
parameter to ε = 0.01.
Paris-Rome
We first consider a synthetic example given by Tversky (1972). It was constructed as
a simple example that the BTL model cannot deal with. We will use this example to
illustrate that the EBA model with infinitely many latent features can recover the latent
structure from choice data.
105